public inbox for [email protected]
help / color / mirror / Atom feedFrom: Sami Imseih <[email protected]>
To: Michael Paquier <[email protected]>
Cc: Andres Freund <[email protected]>
Cc: Robert Haas <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Cc: Thomas Munro <[email protected]>
Subject: Re: dshash_find_or_insert vs. OOM
Date: Tue, 7 Apr 2026 12:00:14 -0500
Message-ID: <CAA5RZ0sX9e2F7ePKxx_805=Wk2BVN2WHhvRhrfd3XBNkvebE9A@mail.gmail.com> (raw)
In-Reply-To: <[email protected]>
References: <CA+TgmoaJwUukUZGu7_yL74oMTQQz2=zqucMhF9+9xBmSC5us1w@mail.gmail.com>
<CAA5RZ0vNVdBnm_TSCR5K6fT7yc7-jQuneNUYFxFgS7hQnn62sA@mail.gmail.com>
<CA+TgmoZ3cd+wh1vz8gB_iD9zxZX558xUERaPm7G-M8wa7p9GNg@mail.gmail.com>
<CAA5RZ0tWjsthTod0ODPEW4noNrSvY+gRWTMxiQ89-yY-k92B8Q@mail.gmail.com>
<CAA5RZ0uhz2uEtUV1KqoCHKQY4U3pgB6N1n-oR-gy8TPUoenQSA@mail.gmail.com>
<CA+TgmoYmzRbNjx8sbm_0Hk-VtLBtJPe=juOUSFkm7ge4Xy7hYw@mail.gmail.com>
<gq55lyolnqm4yv7texawu7sr64btcczktu7hd4byl6zarxkibv@pb6ajailtqop>
<[email protected]>
> On Thu, Mar 26, 2026 at 04:26:33PM -0400, Andres Freund wrote:
> > I think tests like this do have value and I'd definitely run them first while
> > hacking on code related to dshash, rather than relying on the regression tests
> > or such. E.g. having test_aio was invaluable to being able to get AIO into a
> > stable state. When hacking on something with complicated edge cases I'd just
> > add a test for it, making development faster as well as ensuring the
> > complicated case continues to work into the future.
>
> These test modules have a lot of value because they are cheap to run
> and are very usually able to reproduce edge cases that no other place
> of the code tree would be able to reach in a predictible way. Cheap,
> fast and reliable is good. On top of that they can serve as code
> template. Bonus points.
>
> > However, creating its own test module for small parts of the codebase doesn't
> > quite make sense to me. A pretty decent chunk of the test is just boilerplate
> > to add a new module, and every new test module requires its own cluster, which
> > adds a fair bit of runtime overhead, particularly on windows. I think
> > test_dsa, test_dsm_registry, test_dshash should just be one combined test
> > module, they're testing quite closely related code.
>
> Yeah, perhaps grouping all the DSA things into a single module would
> be OK, with a parallel schedule that would speed up things. It
> depends on the complexity and the size of the module to me.
>
> Saying that, I think that the shape of the proposed test_dshash is
> wrong: it proposes one SQL function that does a bunch of
> dshash-related operations in a single function call, in a random
> manner. We have a shared memory state that can survive across SQL
> calls, making it a set of thinner SQL function that wrap directly
> dshash calls able to manipulate the table would feel much more natural
> to me. And it would be easier to design edge cases in the SQL
> scripts themselves.
My apologies for the late response here. I spent some time looking at
this yesterday and came to the conclusion that we can add dshash tests
to test_dsm_registry, which already allocates a dshash via
GetNamedDSHash(). However, I also realized that the API has a gap: callers
cannot set a size limit on the dshash. I need this for the test, but
more importantly it's a limitation of the API itself. So I plan to
target v20 for the tests, as it's likely too late for v19. To start, I've
submitted a patch for allowing callers to set a size limit on a
GetNamedDSHash()-allocated dshash [1].
[1] https://commitfest.postgresql.org/patch/6655/
--
Sami
view thread (15+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: dshash_find_or_insert vs. OOM
In-Reply-To: <CAA5RZ0sX9e2F7ePKxx_805=Wk2BVN2WHhvRhrfd3XBNkvebE9A@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox