pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Andres Freund <andres@anarazel.de>
Cc: Tomas Vondra <tomas.vondra@enterprisedb.com>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: postgres_fdw batching vs. (re)creating the tuple slots
Date: Sun, 30 May 2021 17:10:59 -0400
Message-ID: <2399204.1622409059@sss.pgh.pa.us> (raw)
In-Reply-To: <20210530205831.vib4p7mt23cymhpc@alap3.anarazel.de>
References: <ebbbcc7d-4286-8c28-0272-61b4753af761@enterprisedb.com>
	<20210530205831.vib4p7mt23cymhpc@alap3.anarazel.de>

Andres Freund <andres@anarazel.de> writes:
> On 2021-05-30 22:22:10 +0200, Tomas Vondra wrote:
>> The other problem is that ResourceArrayAdd/Remove seem to behave a bit
>> poorly with very many elements - I'm not sure if it's O(N^2) or worse,
>> but growing the array and linear searches seem to be a bit expensive.

> Hm. I assume this is using the hashed representation of a resowner array
> most of the time, not the array one? I suspect the problem is that
> pretty quickly the ResourceArrayRemove() degrades to a linear search,
> because all of the resowner entries are the same, so the hashing doesn't
> help us at all. The peril of a simplistic open-coded hash table :(

Not only does ResourceArrayRemove degrade, but so does ResourceArrayAdd.

> I think in this specific situation the easiest workaround is to use a
> copy of the tuple desc, instead of the one in the relcache - the copy
> won't be refcounted.

Probably.  There's no obvious reason why these transient slots need
a long-lived tupdesc.  But it does seem like the hashing scheme somebody
added to resowners is a bit too simplistic.  It ought to be able to
cope with lots of refs to the same object, or at least not be extra-awful
for that case.

			regards, tom lane





view thread (15+ messages)  latest in thread

Message-ID: <2399204.1622409059@sss.pgh.pa.us>
Permalink:  ../2399204.1622409059@sss.pgh.pa.us/
Also on:    postgresql.org/message-id/2399204.1622409059@sss.pgh.pa.us

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-hackers@postgresql.org
  Cc: tgl@sss.pgh.pa.us, andres@anarazel.de, tomas.vondra@enterprisedb.com
  Subject: Re: postgres_fdw batching vs. (re)creating the tuple slots
  In-Reply-To: <2399204.1622409059@sss.pgh.pa.us>
  References: <ebbbcc7d-4286-8c28-0272-61b4753af761@enterprisedb.com> <20210530205831.vib4p7mt23cymhpc@alap3.anarazel.de> <2399204.1622409059@sss.pgh.pa.us>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox