MIME-Version: 1.0
References: 
 <CA+TgmoaJwUukUZGu7_yL74oMTQQz2=zqucMhF9+9xBmSC5us1w@mail.gmail.com>
 <CAA5RZ0vNVdBnm_TSCR5K6fT7yc7-jQuneNUYFxFgS7hQnn62sA@mail.gmail.com>
 <CA+TgmoZ3cd+wh1vz8gB_iD9zxZX558xUERaPm7G-M8wa7p9GNg@mail.gmail.com>
 <CAA5RZ0tWjsthTod0ODPEW4noNrSvY+gRWTMxiQ89-yY-k92B8Q@mail.gmail.com>
 <CAA5RZ0uhz2uEtUV1KqoCHKQY4U3pgB6N1n-oR-gy8TPUoenQSA@mail.gmail.com>
 <CA+TgmoYmzRbNjx8sbm_0Hk-VtLBtJPe=juOUSFkm7ge4Xy7hYw@mail.gmail.com>
 <gq55lyolnqm4yv7texawu7sr64btcczktu7hd4byl6zarxkibv@pb6ajailtqop>
 <acXCJODjsCytdpwT@paquier.xyz>
In-Reply-To: <acXCJODjsCytdpwT@paquier.xyz>
From: Sami Imseih <samimseih@gmail.com>
Date: Tue, 7 Apr 2026 12:00:14 -0500
Message-ID: 
 <CAA5RZ0sX9e2F7ePKxx_805=Wk2BVN2WHhvRhrfd3XBNkvebE9A@mail.gmail.com>
Subject: Re: dshash_find_or_insert vs. OOM
To: Michael Paquier <michael@paquier.xyz>
Cc: Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>,
	PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
 Thomas Munro <thomas.munro@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Archived-At: 
 <https://www.postgresql.org/message-id/CAA5RZ0sX9e2F7ePKxx_805%3DWk2BVN2WHhvRhrfd3XBNkvebE9A%40mail.gmail.com>
Precedence: bulk

> On Thu, Mar 26, 2026 at 04:26:33PM -0400, Andres Freund wrote:
> > I think tests like this do have value and I'd definitely run them first while
> > hacking on code related to dshash, rather than relying on the regression tests
> > or such.  E.g. having test_aio was invaluable to being able to get AIO into a
> > stable state.  When hacking on something with complicated edge cases I'd just
> > add a test for it, making development faster as well as ensuring the
> > complicated case continues to work into the future.
>
> These test modules have a lot of value because they are cheap to run
> and are very usually able to reproduce edge cases that no other place
> of the code tree would be able to reach in a predictible way.  Cheap,
> fast and reliable is good.  On top of that they can serve as code
> template.  Bonus points.
>
> > However, creating its own test module for small parts of the codebase doesn't
> > quite make sense to me. A pretty decent chunk of the test is just boilerplate
> > to add a new module, and every new test module requires its own cluster, which
> > adds a fair bit of runtime overhead, particularly on windows.  I think
> > test_dsa, test_dsm_registry, test_dshash should just be one combined test
> > module, they're testing quite closely related code.
>
> Yeah, perhaps grouping all the DSA things into a single module would
> be OK, with a parallel schedule that would speed up things.  It
> depends on the complexity and the size of the module to me.
>
> Saying that, I think that the shape of the proposed test_dshash is
> wrong: it proposes one SQL function that does a bunch of
> dshash-related operations in a single function call, in a random
> manner.  We have a shared memory state that can survive across SQL
> calls, making it a set of thinner SQL function that wrap directly
> dshash calls able to manipulate the table would feel much more natural
> to me.  And it would be easier to design edge cases in the SQL
> scripts themselves.

My apologies for the late response here. I spent some time looking at
this yesterday and came to the conclusion that we can add dshash tests
to test_dsm_registry, which already allocates a dshash via
GetNamedDSHash(). However, I also realized that the API has a gap: callers
cannot set a size limit on the dshash. I need this for the test, but
more importantly it's a limitation of the API itself. So I plan to
target v20 for the tests, as it's likely too late for v19. To start, I've
submitted a patch for allowing callers to set a size limit on a
GetNamedDSHash()-allocated dshash [1].

[1] https://commitfest.postgresql.org/patch/6655/

--
Sami