public inbox for [email protected]help / color / mirror / Atom feed
[PATCH v17 03/10] Add tests on pg_ls_dir before changing it 2+ messages / 2 participants [nested] [flat]
* [PATCH v17 03/10] Add tests on pg_ls_dir before changing it @ 2020-03-17 18:16 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 2+ messages in thread From: Justin Pryzby @ 2020-03-17 18:16 UTC (permalink / raw) --- src/test/regress/expected/misc_functions.out | 18 ++++++++++++++++++ src/test/regress/sql/misc_functions.sql | 5 +++++ 2 files changed, 23 insertions(+) diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index d3acb98d04..2e87c548eb 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -201,6 +201,24 @@ select count(*) > 0 from t (1 row) +select * from (select pg_ls_dir('.', false, true) as name) as ls where ls.name='.'; -- include_dot_dirs=true + name +------ + . +(1 row) + +select * from (select pg_ls_dir('.', false, false) as name) as ls where ls.name='.'; -- include_dot_dirs=false + name +------ +(0 rows) + +select pg_ls_dir('does not exist', true, false); -- ok with missingok=true + pg_ls_dir +----------- +(0 rows) + +select pg_ls_dir('does not exist'); -- fails with missingok=false +ERROR: could not open directory "does not exist": No such file or directory -- -- Test adding a support function to a subject function -- diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql index 094e8f8296..f6857ad177 100644 --- a/src/test/regress/sql/misc_functions.sql +++ b/src/test/regress/sql/misc_functions.sql @@ -60,6 +60,11 @@ select count(*) > 0 from where spcname = 'pg_default') pts join pg_database db on pts.pts = db.oid; +select * from (select pg_ls_dir('.', false, true) as name) as ls where ls.name='.'; -- include_dot_dirs=true +select * from (select pg_ls_dir('.', false, false) as name) as ls where ls.name='.'; -- include_dot_dirs=false +select pg_ls_dir('does not exist', true, false); -- ok with missingok=true +select pg_ls_dir('does not exist'); -- fails with missingok=false + -- -- Test adding a support function to a subject function -- -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0004-Add-pg_ls_dir_metadata-to-list-a-dir-with-file-m.patch" ^ permalink raw reply [nested|flat] 2+ messages in thread
* Large expressions in indexes can't be stored (non-TOASTable) @ 2024-09-03 16:35 Jonathan S. Katz <[email protected]> 0 siblings, 0 replies; 2+ messages in thread From: Jonathan S. Katz @ 2024-09-03 16:35 UTC (permalink / raw) To: PostgreSQL Hackers <[email protected]>; +Cc: Nathan Bossart <[email protected]> Hi, I ran into an issue (previously discussed[1]; quoting Andres out of context that not addressing it then would "[a]ll but guarantee that we'll have this discussion again"[2]) when trying to build a very large expression index that did not fit within the page boundary. The real-world use case was related to a vector search technique where I wanted to use binary quantization based on the relationship between a constant vector (the average at a point-in-time across the entire data set) and the target vector[3][4]. An example: CREATE INDEX ON embeddings USING hnsw((quantization_func(embedding, $VECTOR)) bit_hamming_ops); However, I ran into the issue in[1], where pg_index was identified as catalog that is missing a toast table, even though `indexprs` is marked for extended storage. Providing a very simple reproducer in psql below: ---- CREATE TABLE def (id int); SELECT array_agg(n) b FROM generate_series(1,10_000) n \gset CREATE OR REPLACE FUNCTION vec_quantizer (a int, b int[]) RETURNS bool AS $$ SELECT true $$ LANGUAGE SQL IMMUTABLE; CREATE INDEX ON def (vec_quantizer(id, :'b')); ERROR: row is too big: size 29448, maximum size 8160 --- This can come up with vector searches as vectors can be quite large - the case I was testing involved a 1536-dim floating point vector (~6KB), and the node parse tree pushed past the page boundary by about 2KB. One could argue that pgvector or an extension can build in capabilities to handle quantization internally without requiring the user to provide a source vector (pgvectorscale does this). However, this also limits flexibility to users, as they may want to bring their own quantization functions to vector searches, e.g., as different quantization techniques emerge, or if a particular technique is more suitable for a person's dataset. Thanks, Jonathan [1] https://www.postgresql.org/message-id/flat/84ddff04-f122-784b-b6c5-3536804495f8%40joeconway.com [2] https://www.postgresql.org/message-id/20180720000356.5zkhvfpsqswngyob%40alap3.anarazel.de [3] https://github.com/pgvector/pgvector [4] https://jkatz05.com/post/postgres/pgvector-scalar-binary-quantization/ Attachments: [application/pgp-signature] OpenPGP_signature.asc (840B, ../../[email protected]/2-OpenPGP_signature.asc) download ^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2024-09-03 16:35 UTC | newest] Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2020-03-17 18:16 [PATCH v17 03/10] Add tests on pg_ls_dir before changing it Justin Pryzby <[email protected]> 2024-09-03 16:35 Large expressions in indexes can't be stored (non-TOASTable) Jonathan S. Katz <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox