Message-ID: <41dcba92-1075-e5e5-cb99-36711abf6cec@dunslane.net>
Date: Mon, 4 Dec 2023 09:25:04 -0500
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.15.1
Content-Language: en-US
To: Joe Conway <mail@joeconway.com>, Davin Shearer <davin@apache.org>,
 PostgreSQL-development <pgsql-hackers@postgresql.org>
References: <CALvfUkBxTYy5uWPFVwpk_7ii2zgT07t3d-yR_cy4sfrrLU=kcg@mail.gmail.com>
 <CAFCRh-_LhP5Ln4-beVz=1pQ9VycU3hAxnJHXhPHZRmX0BSCVEQ@mail.gmail.com>
 <CAFCRh--Vhx0105S7SWAYHGcVMTSRtbcoArDUGSbYSWRUGunW_g@mail.gmail.com>
 <CAKFQuwZb3zaSqkOvGXRRLZEW61az+4xyHZ7tBPR91JfrcdCKvA@mail.gmail.com>
 <3853387.1701096982@sss.pgh.pa.us>
 <CAFCRh-_GdiUvjd5z5FfvTfhruOnYqBu163XU47zZE8RNATCJGQ@mail.gmail.com>
 <3a98decf-3fe3-4b49-9b68-fda01338872c@sedlakovi.org>
 <CALvfUkCq3pvkmniWEKZwkbA6hNNvkANF3OamMfhC170RgY+HJg@mail.gmail.com>
 <24e3ee88-ec1e-421b-89ae-8a47ee0d2df1@joeconway.com>
 <b22367ef-6ac1-e1c0-da05-0e760e1b7e29@dunslane.net>
 <CALvfUkD9ZFUU2pe9RMPqdGaKXu5OZi5xzOiPokGhOoPJ56FgAw@mail.gmail.com>
 <7117a356-916c-4cf3-bad8-861490e65dcf@joeconway.com>
 <CALvfUkBKTr9WLAbHQYmwZ7SVp5_VDey1F1uCZtKL4+H1-WvyBw@mail.gmail.com>
 <b604b474-e2b2-4ba0-bc60-fd0cbc05cfc9@joeconway.com>
 <9c77b6fa-ee88-b2e6-0fa7-4fc81721da35@dunslane.net>
 <c30fc5da-0348-4e44-b699-804ccbdcfc36@joeconway.com>
From: Andrew Dunstan <andrew@dunslane.net>
Subject: Re: Emitting JSON to file using COPY TO
In-Reply-To: <c30fc5da-0348-4e44-b699-804ccbdcfc36@joeconway.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Archived-At: <https://www.postgresql.org/message-id/41dcba92-1075-e5e5-cb99-36711abf6cec%40dunslane.net>
Precedence: bulk


On 2023-12-04 Mo 08:37, Joe Conway wrote:
> On 12/4/23 07:41, Andrew Dunstan wrote:
>>
>> On 2023-12-03 Su 20:14, Joe Conway wrote:
>>> (please don't top quote on the Postgres lists)
>>>
>>> On 12/3/23 17:38, Davin Shearer wrote:
>>>> " being quoted as \\" breaks the JSON. It needs to be \".  This has 
>>>> been my whole problem with COPY TO for JSON.
>>>>
>>>> Please validate that the output is in proper format with correct 
>>>> quoting for special characters. I use `jq` on the command line to 
>>>> validate and format the output.
>>>
>>> I just hooked existing "row-to-json machinery" up to the "COPY TO" 
>>> statement. If the output is wrong (just for for this use case?), 
>>> that would be a missing feature (or possibly a bug?).
>>>
>>> Davin -- how did you work around the issue with the way the built in 
>>> functions output JSON?
>>>
>>> Andrew -- comments/thoughts?
>>
>> I meant to mention this when I was making comments yesterday.
>>
>> The patch should not be using CopyAttributeOutText - it will try to
>> escape characters such as \, which produces the effect complained of
>> here, or else we need to change its setup so we have a way to inhibit
>> that escaping.
>
>
> Interesting.
>
> I am surprised this has never been raised as a problem with COPY TO 
> before.
>
> Should the JSON output, as produced by composite_to_json(), be sent 
> as-is with no escaping at all? If yes, is JSON somehow unique in this 
> regard?


Text mode output is in such a form that it can be read back in using 
text mode input. There's nothing special about JSON in this respect - 
any text field will be escaped too. But output suitable for text mode 
input is not what you're trying to produce here; you're trying to 
produce valid JSON.

So, yes, the result of composite_to_json, which is already suitably 
escaped, should not be further escaped in this case.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com