Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lnSiE-0004VQ-Qu for pgsql-hackers@arkaria.postgresql.org; Sun, 30 May 2021 21:11:10 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1lnSiD-0005lR-H2 for pgsql-hackers@arkaria.postgresql.org; Sun, 30 May 2021 21:11:09 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lnSiD-0005lJ-9t for pgsql-hackers@lists.postgresql.org; Sun, 30 May 2021 21:11:09 +0000 Received: from sss.pgh.pa.us ([66.207.139.130]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lnSi8-0000bv-P2 for pgsql-hackers@postgresql.org; Sun, 30 May 2021 21:11:08 +0000 Received: from sss1.sss.pgh.pa.us (localhost [127.0.0.1]) by sss.pgh.pa.us (8.15.2/8.15.2) with ESMTP id 14ULAxjL2399205; Sun, 30 May 2021 17:10:59 -0400 From: Tom Lane To: Andres Freund cc: Tomas Vondra , PostgreSQL-development Subject: Re: postgres_fdw batching vs. (re)creating the tuple slots In-reply-to: <20210530205831.vib4p7mt23cymhpc@alap3.anarazel.de> References: <20210530205831.vib4p7mt23cymhpc@alap3.anarazel.de> Comments: In-reply-to Andres Freund message dated "Sun, 30 May 2021 13:58:31 -0700" MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <2399203.1622409059.1@sss.pgh.pa.us> Date: Sun, 30 May 2021 17:10:59 -0400 Message-ID: <2399204.1622409059@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk Andres Freund writes: > On 2021-05-30 22:22:10 +0200, Tomas Vondra wrote: >> The other problem is that ResourceArrayAdd/Remove seem to behave a bit >> poorly with very many elements - I'm not sure if it's O(N^2) or worse, >> but growing the array and linear searches seem to be a bit expensive. > Hm. I assume this is using the hashed representation of a resowner array > most of the time, not the array one? I suspect the problem is that > pretty quickly the ResourceArrayRemove() degrades to a linear search, > because all of the resowner entries are the same, so the hashing doesn't > help us at all. The peril of a simplistic open-coded hash table :( Not only does ResourceArrayRemove degrade, but so does ResourceArrayAdd. > I think in this specific situation the easiest workaround is to use a > copy of the tuple desc, instead of the one in the relcache - the copy > won't be refcounted. Probably. There's no obvious reason why these transient slots need a long-lived tupdesc. But it does seem like the hashing scheme somebody added to resowners is a bit too simplistic. It ought to be able to cope with lots of refs to the same object, or at least not be extra-awful for that case. regards, tom lane