pg.ddx.io  pgvector/pgvector GitHub issues and pull requests  
help / color / mirror / Atom feed
From: ankane (@ankane) <noreply+ankane@github.com>
To: pgvector/pgvector <noreply+pgvector-pgvector@github.com>
Subject: Re: [pgvector/pgvector] PR #86: Ivf index prefetch
Date: Sun, 21 May 2023 00:19:35 +0000
Message-ID: <gh-pgvector-pgvector-86-c1556049304@github.com> (raw)
In-Reply-To: <gh-pgvector-pgvector-86@github.com>
References: <gh-pgvector-pgvector-86@github.com>

Hi @knizhnik, sorry for not following up sooner. I think the four parts of this are:

1. Changing the buffer access strategy
2. Adding multiple (sequential) pages on inserts 
3. Prefetching pages for `GetScanItems`
4. Prefetching heap pages

For 1, the current strategy was added to prevent [cache wiping](https://momjian.us/main/blogs/pgblog/2020.html#May_13_2020), but from the article, Postgres only uses `BAS_BULKREAD` when it expects to access more than 25% of the shared buffers, so we should probably do the same.

For 2, makes sense to me (related: #133).

Let's break 1 and 2 into separate PRs to discuss.

For 3 and 4, I'm not sure I'd like to add the complexity right now, but to avoid prefetching extra pages with 3, you'll want to use `IVFFLAT_LIST_EXTEND_QUANTUM` for the index build as well. You may also want to use a custom index version and prefetch conditionally to avoid prefetching extra pages with existing indexes.

view thread (11+ messages)  latest in thread

Message-ID: <gh-pgvector-pgvector-86-c1556049304@github.com>
Permalink:  ../gh-pgvector-pgvector-86-c1556049304@github.com/

 · 


This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox