public inbox for [email protected]
help / color / mirror / Atom feedFrom: Melanie Plageman <[email protected]>
To: Thomas Munro <[email protected]>
Cc: Peter Geoghegan <[email protected]>
Cc: Nazir Bilal Yavuz <[email protected]>
Cc: Jonathan S. Katz <[email protected]>
Cc: pgsql-hackers <[email protected]>
Subject: Re: Trying out read streams in pgvector (an extension)
Date: Tue, 9 Dec 2025 16:42:21 -0500
Message-ID: <CAAKRu_ZHvQ=ZFJPzEMS-iPnN_yFmhy1YvpZMRxoDE-eRt6T4-A@mail.gmail.com> (raw)
In-Reply-To: <CA+hUKGJLT2JvWLEiBXMbkSSc5so_Y7=N+S2ce7npjLw8QL3d5w@mail.gmail.com>
References: <CA+hUKGJ_7NKd46nx1wbyXWriuZSNzsTfm+rhEuvU6nxZi3-KVw@mail.gmail.com>
<[email protected]>
<CA+hUKG+x2BcqWzBC77cN0ewhzMF0kYhC6c4G_T2gJLPbqYQ6Ow@mail.gmail.com>
<CA+hUKGL-3mBtkA9RTbLFHuSS5cviuv0ko7nBhCg9KM7Q-GSEkw@mail.gmail.com>
<CAAKRu_ZVxzwRRbxedgb_LtkFaGf78XAbTO9uExvadV2DzaE=Jg@mail.gmail.com>
<CA+hUKG+zLmkD9zus=JOjjC+j5p9R1+CSXNZgd5=exZ01ZTaKoA@mail.gmail.com>
<CA+hUKGJx6FNqzsxfSOGH0nJZJq1MBc+t7NBKtAmy6zj4HD86tA@mail.gmail.com>
<CAN55FZ16TEhgYbK=qSEbkO8utz+u232NksCEmJMC1G4iZvnbvA@mail.gmail.com>
<CA+hUKGL7-Dx8KiUo=G91Y5tfFpwDUFFQJ6=9D8Gr1n=DZxGh+w@mail.gmail.com>
<CAAKRu_ZGhnWZXOyEyZ2r47g-F7U8asMRA6U8YZw3h=2rR=m_hQ@mail.gmail.com>
<CAN55FZ0tgjF1beJSRXw3rgkbzwPZ7ngChJkPZm9aJkPuaF=dmg@mail.gmail.com>
<CAAKRu_Zwj83zCJhahhMO578-+JdfTbqMV_ktxr-XjiE8BHLo9g@mail.gmail.com>
<CA+hUKGJLT2JvWLEiBXMbkSSc5so_Y7=N+S2ce7npjLw8QL3d5w@mail.gmail.com>
On Mon, Dec 8, 2025 at 10:47 PM Thomas Munro <[email protected]> wrote:
>
> I think it'd be better if that were the consumer's choice. I don't
> want the consumer to be required to drain the stream before resuming,
> as that'd be an unprincipled stall. For example, if new WAL arrives
> over the network then I think it should be possible for recovery's
> WAL-powered stream of heap pages to resume looking ahead even if
> recovery hasn't drained the existing stream completely.
>
> 1. read_stream_resume() as before, but with a new explicit
> read_stream_pause(): if a block number callback would like to report a
> temporary lack of information, it should return
> read_stream_pause(stream), not InvalidBlockNumber. Then after
> read_stream_resume(stream) is called, the next
> read_stream_next_buffer() enters the lookahead loop again. While
> paused, if the consumer drains all the existing buffers in the stream
> and then one more, it will receive InvalidBuffer, but if the _resume()
> call is made sooner, the consumer won't ever know about the temporary
> lack of buffers in the stream.
I like this new interface. If the user does want to exhaust the stream
(as was the case with earlier pgvector read stream user code), I
assume you would want to do:
read_stream_pause()
read_stream_reset()
read_stream_resume()
- Melanie
view thread (18+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: Trying out read streams in pgvector (an extension)
In-Reply-To: <CAAKRu_ZHvQ=ZFJPzEMS-iPnN_yFmhy1YvpZMRxoDE-eRt6T4-A@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox