Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qTQi2-009Yk2-Qr for pgsql-hackers@arkaria.postgresql.org; Tue, 08 Aug 2023 17:41:31 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1qTQi0-000265-Al for pgsql-hackers@arkaria.postgresql.org; Tue, 08 Aug 2023 17:41:28 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qTQhy-00025x-TO for pgsql-hackers@lists.postgresql.org; Tue, 08 Aug 2023 17:41:28 +0000 Received: from wout2-smtp.messagingengine.com ([64.147.123.25]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qTQhs-001KOC-Lo for pgsql-hackers@lists.postgresql.org; Tue, 08 Aug 2023 17:41:26 +0000 Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.west.internal (Postfix) with ESMTP id D9EF43200319; Tue, 8 Aug 2023 13:41:14 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Tue, 08 Aug 2023 13:41:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1691516474; x= 1691602874; bh=L/FWbrgoeARV5JCq3xGo9nhTzRHSdWMZdMLiRl4SL60=; b=P aYoHfPMJki6gSXg9OWXk7PQ6Gw/g/Jx17wE+w1Hcr7q++jTkisf+k8PRF1/tnG8O dmfxRVSJkhH4oQMbb8+9Co4hb2SYNj3wMBhdJi4RBxf9Ps15NMZwuio/mmZw4LiK A9l5oL5SPx/s2vgnivXe7lM4agLz87jX1nNhkxfQWmMwJgA+LOQbgHSB/ow/7uwK vZC/U6NPXh8nACG5Ulla4xX+2P/0qaeGV/1+qCSWhtXG4kTJZ8+aIjwHpZslgmPF +Ux3jIC/ftplEFumxzNYkKXfQ4GEGkgh9Uepi4I7vPSRNY268v/cRELdULew0IMq 6M9BjLMQN5+g3xtM2+AEQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrledvgdduudegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkgggtugfgjgesmhekreertddtjeenucfhrhhomheptehlvhgr rhhoucfjvghrrhgvrhgruceorghlvhhhvghrrhgvsegrlhhvhhdrnhhoqdhiphdrohhrgh eqnecuggftrfgrthhtvghrnhepudelkeekgfdttdettdekgfduvdeiffelteegjeeihfet heekveeftedtuddvhfeunecuffhomhgrihhnpegvnhhtvghrphhrihhsvggusgdrtghomh enucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegrlhhv hhgvrhhrvgesrghlvhhhrdhnohdqihhprdhorhhg X-ME-Proxy: Feedback-ID: ia2694551:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 8 Aug 2023 13:41:13 -0400 (EDT) Received: by perhan.alvh.no-ip.org (Postfix, from userid 1000) id A3D12EA; Tue, 8 Aug 2023 19:41:10 +0200 (CEST) Date: Tue, 8 Aug 2023 19:41:10 +0200 From: Alvaro Herrera To: Amit Langote Cc: pgsql-hackers@lists.postgresql.org Subject: Re: pgsql: Some refactoring to export json(b) conversion functions Message-ID: <20230808174110.oq3iymllsv6amkih@alvherre.pgsql> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="lzw6hdh7nrlzilxo" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --lzw6hdh7nrlzilxo Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit On 2023-Jul-26, Amit Langote wrote: > Some refactoring to export json(b) conversion functions > > This is to export datum_to_json(), datum_to_jsonb(), and > jsonb_from_cstring(), though the last one is exported as > jsonb_from_text(). After this commit, Coverity started complaining that datum_to_jsonb_internal() leaks the JsonLexContext here 754 │ case JSONTYPE_CAST: 755 │ case JSONTYPE_JSON: 756 │ { 757 │ /* parse the json right into the existing result object */ 758 │ JsonLexContext *lex; 759 │ JsonSemAction sem; 760 │ text *json = DatumGetTextPP(val); 761 │ 762 │ lex = makeJsonLexContext(json, true); 763 │ 764 │ memset(&sem, 0, sizeof(sem)); 765 │ 766 │ sem.semstate = (void *) result; 767 │ 768 │ sem.object_start = jsonb_in_object_start; 769 │ sem.array_start = jsonb_in_array_start; 770 │ sem.object_end = jsonb_in_object_end; 771 │ sem.array_end = jsonb_in_array_end; 772 │ sem.scalar = jsonb_in_scalar; 773 │ sem.object_field_start = jsonb_in_object_field_start; 774 │ 775 │ pg_parse_json_or_ereport(lex, &sem); 776 │ } 777 │ break; Admittedly, our support code for this is not great, since we have no clean way to free those resources. Some places like json_object_keys are freeing everything manually (even though in that particular case it's unnecessary, since that one runs in a memcxt that's going to be cleaned up shortly afterwards). One idea that Tom floated was to allow the JsonLexContext to be optionally stack-allocated. That reduces palloc() traffic; but some callers do need it to be palloc'ed. Here's a patch that does it that way, and adds a freeing routine that knows what to do in either case. It may make sense to do some further analysis and remove useless free calls. It may make sense to change the structs that contain JsonLexContext * so that they directly embed JsonLexContext instead. That would further reduce palloc'ing. -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/ "Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end." (2nd Commandment for C programmers) --lzw6hdh7nrlzilxo Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0001-JsonLexContext-allocation-free.patch"