From: Andrew Dunstan <andrew@dunslane.net>
To: Joe Conway <mail@joeconway.com>
To: Davin Shearer <davin@apache.org>
To: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
Subject: Re: Emitting JSON to file using COPY TO
Date: Sun, 3 Dec 2023 10:10:38 -0500
Message-ID: <b22367ef-6ac1-e1c0-da05-0e760e1b7e29@dunslane.net> (raw)
In-Reply-To: <24e3ee88-ec1e-421b-89ae-8a47ee0d2df1@joeconway.com>
References: <CALvfUkBxTYy5uWPFVwpk_7ii2zgT07t3d-yR_cy4sfrrLU=kcg@mail.gmail.com>
<fc1e39f8-274f-4be6-8a70-9fb0d9fe84a2@aklaver.com>
<CAFCRh-_LhP5Ln4-beVz=1pQ9VycU3hAxnJHXhPHZRmX0BSCVEQ@mail.gmail.com>
<CAFCRh--Vhx0105S7SWAYHGcVMTSRtbcoArDUGSbYSWRUGunW_g@mail.gmail.com>
<CAKFQuwZb3zaSqkOvGXRRLZEW61az+4xyHZ7tBPR91JfrcdCKvA@mail.gmail.com>
<3853387.1701096982@sss.pgh.pa.us>
<CAFCRh-_GdiUvjd5z5FfvTfhruOnYqBu163XU47zZE8RNATCJGQ@mail.gmail.com>
<3a98decf-3fe3-4b49-9b68-fda01338872c@sedlakovi.org>
<CALvfUkCq3pvkmniWEKZwkbA6hNNvkANF3OamMfhC170RgY+HJg@mail.gmail.com>
<24e3ee88-ec1e-421b-89ae-8a47ee0d2df1@joeconway.com>
On 2023-12-01 Fr 14:28, Joe Conway wrote:
> On 11/29/23 10:32, Davin Shearer wrote:
>> Thanks for the responses everyone.
>>
>> I worked around the issue using the `psql -tc` method as Filip
>> described.
>>
>> I think it would be great to support writing JSON using COPY TO at
>> some point so I can emit JSON to files using a PostgreSQL function
>> directly.
>>
>> -Davin
>>
>> On Tue, Nov 28, 2023 at 2:36 AM Filip Sedlák <filip@sedlakovi.org
>> <mailto:filip@sedlakovi.org>> wrote:
>>
>> This would be a very special case for COPY. It applies only to a
>> single
>> column of JSON values. The original problem can be solved with psql
>> --tuples-only as David wrote earlier.
>>
>>
>> $ psql -tc 'select json_agg(row_to_json(t))
>> from (select * from public.tbl_json_test) t;'
>>
>> [{"id":1,"t_test":"here's a \"string\""}]
>>
>>
>> Special-casing any encoding/escaping scheme leads to bugs and harder
>> parsing.
>
> (moved to hackers)
>
> I did a quick PoC patch (attached) -- if there interest and no hard
> objections I would like to get it up to speed for the January commitfest.
>
> Currently the patch lacks documentation and regression test support.
>
> Questions:
> ----------
> 1. Is supporting JSON array format sufficient, or does it need to
> support some other options? How flexible does the support scheme need
> to be?
>
> 2. This only supports COPY TO and we would undoubtedly want to support
> COPY FROM for JSON as well, but is that required from the start?
>
> Thanks for any feedback.
I realize this is just a POC, but I'd prefer to see composite_to_json()
not exposed. You could use the already public datum_to_json() instead,
passing JSONTYPE_COMPOSITE and F_RECORD_OUT as the second and third
arguments.
I think JSON array format is sufficient.
I can see both sides of the COPY FROM argument, but I think insisting on
that makes this less doable for release 17. On balance I would stick to
COPY TO for now.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-general@postgresql.org
Cc: andrew@dunslane.net, mail@joeconway.com, davin@apache.org, pgsql-hackers@postgreSQL.org
Subject: Re: Emitting JSON to file using COPY TO
In-Reply-To: <b22367ef-6ac1-e1c0-da05-0e760e1b7e29@dunslane.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox