public inbox for [email protected]
help / color / mirror / Atom feedFrom: Junwang Zhao <[email protected]>
To: jian he <[email protected]>
Cc: Florents Tselai <[email protected]>
Cc: Joe Conway <[email protected]>
Cc: Andrey M. Borodin <[email protected]>
Cc: Dean Rasheed <[email protected]>
Cc: Daniel Verite <[email protected]>
Cc: Andrew Dunstan <[email protected]>
Cc: Davin Shearer <[email protected]>
Cc: PostgreSQL-development <[email protected]>
Subject: Re: Emitting JSON to file using COPY TO
Date: Sat, 7 Feb 2026 21:27:29 +0800
Message-ID: <CAEG8a3LB7q1eQ0AzFTEBDFDxs3kJ=5iJA+HTmiYGya6Wb5jsRA@mail.gmail.com> (raw)
In-Reply-To: <CACJufxEjZwrocCpt29xtmJTwhWZUu1Nt0GfHFarNGXUS5AAecw@mail.gmail.com>
References: <[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
<CAEZATCWh29787xf=4NgkoixeqRHrqi0Qd33Z6_-F8t2dZ0yLCQ@mail.gmail.com>
<[email protected]>
<[email protected]>
<[email protected]>
<CACJufxGMSbpXCCoCb_u5EULo5y2sfMgASkRk2pmw9WRu2qOCXw@mail.gmail.com>
<CACJufxGjJXKfPZoyFqgvYTBYW3GmB5WA+0H0zXkfYjrc7au7qg@mail.gmail.com>
<CACJufxET6LJWbywU_odC_k3vYwEKhNQjiWTDk=37ecQG5mFk5Q@mail.gmail.com>
<CACJufxFvOQWcGpiaFkOxMGPc=g85h=-LdyB_2wXBSVQCO9RAHQ@mail.gmail.com>
<CACJufxH=3PreKDHuoskn0GQmxUcw-n0k3ogN1CD4CvFbLBy-VA@mail.gmail.com>
<CACJufxH8J0uD-inukxAmd3TVwt-b-y7d7hLGSBdEdLXFGJLyDA@mail.gmail.com>
<CACJufxG_VY-Hv6ss8pqDxP8SaBmBPYUdv=LB3QFpsqtTZu9FYQ@mail.gmail.com>
<CAEG8a3J=Wb0dv4uwhhMXQBNPfZ63_ZaSdko8-chK-tc-wSAwog@mail.gmail.com>
<CACJufxGmSw5GUOquAT+q7B0k+xweU3NwcNc53fLjHQjsMeanaw@mail.gmail.com>
<CACJufxGK1WiwN48Srr6Si_XY4kz7SmDKBGxgyR3EE2WoeXJgFA@mail.gmail.com>
<CACJufxGF0F8ooTCCaY=PLhb-K84NOz=SYGjYF8ZrbgO8C0iFqQ@mail.gmail.com>
<CACJufxGbiNtsHnn=ZGw2J4VcjD5za5znbcxsaEvVd7vMdxQQ+g@mail.gmail.com>
<CACJufxG7T2_LF=UuJ4iFYMJZsK37L6d3XJvBrcR-3Pp3z+BiGw@mail.gmail.com>
<CACJufxG0=CoG64Ng7DfMP4zy0MORhTW7rRioQnNVCw-6GxRaXg@mail.gmail.com>
<CACJufxGq6aJszBhfrUQKHeKmp0r+ka7=SwHSY7Qs_6LQMwiEbw@mail.gmail.com>
<CA+v5N43-F9-Wiktg_-+aqKJz+YiCNJMAh5ootDeTKkOJ=kiaHA@mail.gmail.com>
<CACJufxEjZwrocCpt29xtmJTwhWZUu1Nt0GfHFarNGXUS5AAecw@mail.gmail.com>
Hi jian,
Thanks for keeping the patch set up to date with the master.
On Fri, Feb 6, 2026 at 11:26 AM jian he <[email protected]> wrote:
>
> On Wed, Feb 4, 2026 at 12:41 AM Florents Tselai
> <[email protected]> wrote:
> >
> > I (and others I assume) would really like to see this in 19;
> > glancing at the thread above and in the test cases I see this is in good shape for comitter review.
> > No?
> >
> > If I were to add something it would be an example in copy.sgml
> > <para>
> > When the <literal>FORCE_ARRAY</literal> option is enabled,
> > the entire output is wrapped in a JSON array and individual rows are separated by commas:
> > <programlisting>
> > COPY (SELECT id, name FROM users) TO STDOUT (FORMAT JSON, FORCE_ARRAY);
> > </programlisting>
> > <programlisting>
> > [
> > {"id": 1, "name": "Alice"}
> > ,{"id": 2, "name": "Bob"}
> > ,{"id": 3, "name": "Charlie"}
> > ]
> > </programlisting>
> > </para>
> >
>
> v23-0003-Add-option-force_array-for-COPY-JSON-FORMAT.patch
> I've added:
>
> +<para>
> + When the <literal>FORCE_ARRAY</literal> option is enabled,
> + the entire output is wrapped in a single JSON array with rows
> separated by commas:
> +<programlisting>
> +COPY (SELECT * FROM (VALUES(1),(2)) val(id)) TO STDOUT (FORMAT JSON,
> FORCE_ARRAY);
> +</programlisting>
> +The output is as follows:
> +<screen>
> +[
> + {"id":1}
> +,{"id":2}
> +]
> +</screen>
> +</para>
> +
> +
>
> > Also, apologies if that has been discussed already,
> > is there a good reason why didn't we just go with a simple "WRAP_ARRAY" ?
> >
>
> I don’t have a particular preference.
> If the consensus is that WRAP_ARRAY is better than FORCE_ARRAY, we can
> change it accordingly.
>
>
> --
> jian
> https://www.enterprisedb.com/
Here are some comments on v23:
0001: The refactor looks straightforward to me. Introducing a format
field should make future extensions easier. One suggestion is that we
could add some helper macros around format, for example:
#define IS_FORMAT_CSV(format) (format == COPY_FORMAT_CSV)
#define IS_FORMAT_TEXT_LIKE(format) \
(format == COPY_FORMAT_TEXT || format == COPY_FORMAT_CSV)
I think this would improve readability.
0002: Since you have moved the `CopyFormat enum` into 0001, the
following commit msg should be rephrased.
The CopyFormat enum was originally contributed by Joel Jacobson
[email protected], later refactored by Jian He to address various issues, and
further adapted by Junwang Zhao to support the newly introduced CopyToRoutine
struct (commit 2e4127b6d2).
- if (opts_out->format == COPY_FORMAT_BINARY && opts_out->delim)
- ereport(ERROR,
- (errcode(ERRCODE_SYNTAX_ERROR),
- /*- translator: %s is the name of a COPY option, e.g. ON_ERROR */
- errmsg("cannot specify %s in BINARY mode", "DELIMITER")));
+ if (opts_out->delim)
+ {
+ if (opts_out->format == COPY_FORMAT_BINARY)
+ ereport(ERROR,
+ errcode(ERRCODE_SYNTAX_ERROR),
+ /*- translator: %s is the name of a COPY option, e.g. ON_ERROR */
+ errmsg("cannot specify %s in BINARY mode", "DELIMITER"));
+ else if (opts_out->format == COPY_FORMAT_JSON)
+ ereport(ERROR,
+ errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("cannot specify %s in JSON mode", "DELIMITER"));
+ }
Can we add a function that converts CopyFormat to a string? Treating
CopyFormat as %s in error messages would make the code shorter.
However, I'm not sure whether this aligns with translation
conventions, correct me if I'm wrong.
- * CSV and text formats share the same TextLike routines except for the
+ * CSV and text, json formats share the same TextLike routines except for the
I'd suggest rewording to `CSV, text and json ...`. The same applied to
other parts in this patch.
0003: The commit message includes some changes(adapt the newly
introduced CopyToRoutine) that actually belong to 0002; it would be
better to remove them from this commit.
+ if (cstate->json_row_delim_needed && cstate->opts.force_array)
+ CopySendChar(cstate, ',');
+ else if (cstate->opts.force_array)
+ {
+ /* first row needs no delimiter */
+ CopySendChar(cstate, ' ');
+ cstate->json_row_delim_needed = true;
+ }
can we do this:
if (cstate->opts.force_array)
{
if (cstate->json_row_delim_needed)
CopySendChar(cstate, ',');
else
{
/* first row needs no delimiter */
CopySendChar(cstate, ' ');
cstate->json_row_delim_needed = true;
}
}
--
Regards
Junwang Zhao
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: Emitting JSON to file using COPY TO
In-Reply-To: <CAEG8a3LB7q1eQ0AzFTEBDFDxs3kJ=5iJA+HTmiYGya6Wb5jsRA@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox