public inbox for [email protected]help / color / mirror / Atom feed
pgsql: Keep the decompressed filter in brin_bloom_union 5+ messages / 1 participants [nested] [flat]
* pgsql: Keep the decompressed filter in brin_bloom_union @ 2025-03-26 16:06 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 5+ messages in thread From: Tomas Vondra @ 2025-03-26 16:06 UTC (permalink / raw) To: [email protected] Keep the decompressed filter in brin_bloom_union The brin_bloom_union() function combines two BRIN summaries, by merging one filter into the other. With bloom, we have to decompress the filters first, but the function failed to update the summary to store the merged filter. As a consequence, the index may be missing some of the data, and return false negatives. This issue exists since BRIN bloom indexes were introduced in Postgres 14, but at that point the union function was called only when two sessions happened to summarize a range concurrently, which is rare. It got much easier to hit in 17, as parallel builds use the union function to merge summaries built by workers. Fixed by storing a pointer to the decompressed filter, and freeing the original one. Free the second filter too, if it was decompressed. The freeing is not strictly necessary, because the union is called in short-lived contexts, but it's tidy. Backpatch to 14, where BRIN bloom indexes were introduced. Reported by Arseniy Mukhin, investigation and fix by me. Reported-by: Arseniy Mukhin Discussion: https://postgr.es/m/18855-1cf1c8bcc22150e6%40postgresql.org Backpatch-through: 14 Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/818245506c285e0325141dabb1ced45057937502 Modified Files -------------- src/backend/access/brin/brin_bloom.c | 11 +++++++++++ 1 file changed, 11 insertions(+) ^ permalink raw reply [nested|flat] 5+ messages in thread
* pgsql: Keep the decompressed filter in brin_bloom_union @ 2025-03-26 16:06 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 5+ messages in thread From: Tomas Vondra @ 2025-03-26 16:06 UTC (permalink / raw) To: [email protected] Keep the decompressed filter in brin_bloom_union The brin_bloom_union() function combines two BRIN summaries, by merging one filter into the other. With bloom, we have to decompress the filters first, but the function failed to update the summary to store the merged filter. As a consequence, the index may be missing some of the data, and return false negatives. This issue exists since BRIN bloom indexes were introduced in Postgres 14, but at that point the union function was called only when two sessions happened to summarize a range concurrently, which is rare. It got much easier to hit in 17, as parallel builds use the union function to merge summaries built by workers. Fixed by storing a pointer to the decompressed filter, and freeing the original one. Free the second filter too, if it was decompressed. The freeing is not strictly necessary, because the union is called in short-lived contexts, but it's tidy. Backpatch to 14, where BRIN bloom indexes were introduced. Reported by Arseniy Mukhin, investigation and fix by me. Reported-by: Arseniy Mukhin Discussion: https://postgr.es/m/18855-1cf1c8bcc22150e6%40postgresql.org Backpatch-through: 14 Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/cb0ad70b8e47a2beb199b2106fb652f6a287aade Modified Files -------------- src/backend/access/brin/brin_bloom.c | 11 +++++++++++ 1 file changed, 11 insertions(+) ^ permalink raw reply [nested|flat] 5+ messages in thread
* pgsql: Keep the decompressed filter in brin_bloom_union @ 2025-03-26 16:06 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 5+ messages in thread From: Tomas Vondra @ 2025-03-26 16:06 UTC (permalink / raw) To: [email protected] Keep the decompressed filter in brin_bloom_union The brin_bloom_union() function combines two BRIN summaries, by merging one filter into the other. With bloom, we have to decompress the filters first, but the function failed to update the summary to store the merged filter. As a consequence, the index may be missing some of the data, and return false negatives. This issue exists since BRIN bloom indexes were introduced in Postgres 14, but at that point the union function was called only when two sessions happened to summarize a range concurrently, which is rare. It got much easier to hit in 17, as parallel builds use the union function to merge summaries built by workers. Fixed by storing a pointer to the decompressed filter, and freeing the original one. Free the second filter too, if it was decompressed. The freeing is not strictly necessary, because the union is called in short-lived contexts, but it's tidy. Backpatch to 14, where BRIN bloom indexes were introduced. Reported by Arseniy Mukhin, investigation and fix by me. Reported-by: Arseniy Mukhin Discussion: https://postgr.es/m/18855-1cf1c8bcc22150e6%40postgresql.org Backpatch-through: 14 Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/ebcc799a772ee0097ba86e9ae63481dd44277a77 Modified Files -------------- src/backend/access/brin/brin_bloom.c | 11 +++++++++++ 1 file changed, 11 insertions(+) ^ permalink raw reply [nested|flat] 5+ messages in thread
* pgsql: Keep the decompressed filter in brin_bloom_union @ 2025-03-26 16:06 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 5+ messages in thread From: Tomas Vondra @ 2025-03-26 16:06 UTC (permalink / raw) To: [email protected] Keep the decompressed filter in brin_bloom_union The brin_bloom_union() function combines two BRIN summaries, by merging one filter into the other. With bloom, we have to decompress the filters first, but the function failed to update the summary to store the merged filter. As a consequence, the index may be missing some of the data, and return false negatives. This issue exists since BRIN bloom indexes were introduced in Postgres 14, but at that point the union function was called only when two sessions happened to summarize a range concurrently, which is rare. It got much easier to hit in 17, as parallel builds use the union function to merge summaries built by workers. Fixed by storing a pointer to the decompressed filter, and freeing the original one. Free the second filter too, if it was decompressed. The freeing is not strictly necessary, because the union is called in short-lived contexts, but it's tidy. Backpatch to 14, where BRIN bloom indexes were introduced. Reported by Arseniy Mukhin, investigation and fix by me. Reported-by: Arseniy Mukhin Discussion: https://postgr.es/m/18855-1cf1c8bcc22150e6%40postgresql.org Backpatch-through: 14 Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/e064b770c081c14703ff494893de1ec556d79b69 Modified Files -------------- src/backend/access/brin/brin_bloom.c | 11 +++++++++++ 1 file changed, 11 insertions(+) ^ permalink raw reply [nested|flat] 5+ messages in thread
* pgsql: Keep the decompressed filter in brin_bloom_union @ 2025-03-26 16:06 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 5+ messages in thread From: Tomas Vondra @ 2025-03-26 16:06 UTC (permalink / raw) To: [email protected] Keep the decompressed filter in brin_bloom_union The brin_bloom_union() function combines two BRIN summaries, by merging one filter into the other. With bloom, we have to decompress the filters first, but the function failed to update the summary to store the merged filter. As a consequence, the index may be missing some of the data, and return false negatives. This issue exists since BRIN bloom indexes were introduced in Postgres 14, but at that point the union function was called only when two sessions happened to summarize a range concurrently, which is rare. It got much easier to hit in 17, as parallel builds use the union function to merge summaries built by workers. Fixed by storing a pointer to the decompressed filter, and freeing the original one. Free the second filter too, if it was decompressed. The freeing is not strictly necessary, because the union is called in short-lived contexts, but it's tidy. Backpatch to 14, where BRIN bloom indexes were introduced. Reported by Arseniy Mukhin, investigation and fix by me. Reported-by: Arseniy Mukhin Discussion: https://postgr.es/m/18855-1cf1c8bcc22150e6%40postgresql.org Backpatch-through: 14 Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/6be02bbc884b1dd0e0791fc7c7d8280eb2078fce Modified Files -------------- src/backend/access/brin/brin_bloom.c | 11 +++++++++++ 1 file changed, 11 insertions(+) ^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2025-03-26 16:06 UTC | newest] Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2025-03-26 16:06 pgsql: Keep the decompressed filter in brin_bloom_union Tomas Vondra <[email protected]> 2025-03-26 16:06 pgsql: Keep the decompressed filter in brin_bloom_union Tomas Vondra <[email protected]> 2025-03-26 16:06 pgsql: Keep the decompressed filter in brin_bloom_union Tomas Vondra <[email protected]> 2025-03-26 16:06 pgsql: Keep the decompressed filter in brin_bloom_union Tomas Vondra <[email protected]> 2025-03-26 16:06 pgsql: Keep the decompressed filter in brin_bloom_union Tomas Vondra <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox