MIME-Version: 1.0
References: <7b0a51d6-0d9d-7366-3a1a-f74397a02f55@dunslane.net>
 <CA+TgmoZKqZ+xXN550e2DXM-F4ELCE+Ee=Ogy4pZZMhu9SBRmzQ@mail.gmail.com> <253cef9e-1360-a1a6-5091-ad4368b1b8cc@dunslane.net>
In-Reply-To: <253cef9e-1360-a1a6-5091-ad4368b1b8cc@dunslane.net>
From: Robert Haas <robertmhaas@gmail.com>
Date: Wed, 3 Jan 2024 08:45:06 -0500
Message-ID: <CA+TgmoYLi7RjjbWvBHa_9+2rVTuO4C1xPHJqzCdNUnHxR3-NdA@mail.gmail.com>
Subject: Re: WIP Incremental JSON Parser
To: Andrew Dunstan <andrew@dunslane.net>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://www.postgresql.org/message-id/CA%2BTgmoYLi7RjjbWvBHa_9%2B2rVTuO4C1xPHJqzCdNUnHxR3-NdA%40mail.gmail.com>
Precedence: bulk

On Wed, Jan 3, 2024 at 6:57=E2=80=AFAM Andrew Dunstan <andrew@dunslane.net>=
 wrote:
> Yeah. One idea I had yesterday was to stash the field names, which in
> large JSON docs tent to be pretty repetitive, in a hash table instead of
> pstrduping each instance. The name would be valid until the end of the
> parse, and would only need to be duplicated by the callback function if
> it were needed beyond that. That's not the case currently with the
> parse_manifest code. I'll work on using a hash table.

IMHO, this is not a good direction. Anybody who is parsing JSON
probably wants to discard the duplicated labels and convert other
heavily duplicated strings to enum values or something. (e.g. if every
record has {"color":"red"} or {"color":"green"}). So the hash table
lookups will cost but won't really save anything more than just
freeing the memory not needed, but will probably be more expensive.

> The parse_manifest code does seem to pfree the scalar values it no
> longer needs fairly well, so maybe we don't need to to anything there.

Hmm. This makes me wonder if you've measured how much actual leakage there =
is?

--=20
Robert Haas
EDB: http://www.enterprisedb.com