agora inbox for pgsql-committers@postgresql.orghelp / color / mirror / Atom feed
pgsql: Harden tsvector code against overflows. 6+ messages / 1 participants [nested] [flat]
* pgsql: Harden tsvector code against overflows. @ 2026-08-10 13:41 Noah Misch <noah@leadboat.com> 0 siblings, 0 replies; 6+ messages in thread From: Noah Misch @ 2026-08-10 13:41 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Harden tsvector code against overflows. The core of this patch is to prevent array_to_tsvector() from generating invalid tsvectors. It did not check for overly-long lexemes (so that WordEntry.len fields could overflow), nor did it check that the total "datalen" fits within MAXSTRPOS (so that WordEntry.pos fields could overflow, and the number of entries in the tsvector could be much more than the normal limit). While the field overflows couldn't do anything much worse than produce a corrupted tsvector value, a sufficiently large number of tsvector entries could cause integer overflows in later processing, such as tsvectorout. Another important fix is to prevent tsvectorrecv() from accepting invalid tsvectors. The main problem there is that it did not reject empty-string lexemes. Hence, even though it did (mostly) enforce the MAXSTRPOS limit, it could still produce a result with an unreasonable number of tsvector entries, if they were primarily empty strings. Also, fix tsvectorout's calculation of its required output buffer size: it was multiplying the string lengths by pg_database_encoding_max_length() for no reason. That contributed to the risk of integer overflow there. With valid tsvector input, there's no risk, but there's still no reason to make the output buffer several times bigger than needed. I also tried to make a couple of related routines more robust, and spent some effort on improving the comments in ts_type.h. Also, standardize on a single spelling of the "string is too long for tsvector" message, using %zu instead of an assortment of formats. These changes aren't security per se but came out of inspecting the code for problems. Reported-by: Yuhang Wu <yuhang@depthfirst.com> and Zhenpeng Lin Reported-by: Zheng Yu <zheng@depthfirst.com> Reported-by: Hcamael <baiyjrh@gmail.com> Author: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Amit Langote <amitlangote09@gmail.com> Backpatch-through: 14 Security: CVE-2026-14662 Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/23d9ad771817c123ebdc16ebc65977754c82b5ab Author: Tom Lane <tgl@sss.pgh.pa.us> Modified Files -------------- src/backend/tsearch/to_tsany.c | 23 +++++++++++++++----- src/backend/tsearch/ts_parse.c | 27 ++++++++++++++++++----- src/backend/utils/adt/tsvector.c | 28 ++++++++++++++++++------ src/backend/utils/adt/tsvector_op.c | 40 +++++++++++++++++++++++++++++----- src/include/tsearch/ts_type.h | 43 ++++++++++++++++++++++++++----------- 5 files changed, 126 insertions(+), 35 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Harden tsvector code against overflows. @ 2026-08-10 13:41 Noah Misch <noah@leadboat.com> 0 siblings, 0 replies; 6+ messages in thread From: Noah Misch @ 2026-08-10 13:41 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Harden tsvector code against overflows. The core of this patch is to prevent array_to_tsvector() from generating invalid tsvectors. It did not check for overly-long lexemes (so that WordEntry.len fields could overflow), nor did it check that the total "datalen" fits within MAXSTRPOS (so that WordEntry.pos fields could overflow, and the number of entries in the tsvector could be much more than the normal limit). While the field overflows couldn't do anything much worse than produce a corrupted tsvector value, a sufficiently large number of tsvector entries could cause integer overflows in later processing, such as tsvectorout. Another important fix is to prevent tsvectorrecv() from accepting invalid tsvectors. The main problem there is that it did not reject empty-string lexemes. Hence, even though it did (mostly) enforce the MAXSTRPOS limit, it could still produce a result with an unreasonable number of tsvector entries, if they were primarily empty strings. Also, fix tsvectorout's calculation of its required output buffer size: it was multiplying the string lengths by pg_database_encoding_max_length() for no reason. That contributed to the risk of integer overflow there. With valid tsvector input, there's no risk, but there's still no reason to make the output buffer several times bigger than needed. I also tried to make a couple of related routines more robust, and spent some effort on improving the comments in ts_type.h. Also, standardize on a single spelling of the "string is too long for tsvector" message, using %zu instead of an assortment of formats. These changes aren't security per se but came out of inspecting the code for problems. Reported-by: Yuhang Wu <yuhang@depthfirst.com> and Zhenpeng Lin Reported-by: Zheng Yu <zheng@depthfirst.com> Reported-by: Hcamael <baiyjrh@gmail.com> Author: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Amit Langote <amitlangote09@gmail.com> Backpatch-through: 14 Security: CVE-2026-14662 Branch ------ REL_19_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/cb947ca31f6947f3746341a635f90395a3ee52e6 Author: Tom Lane <tgl@sss.pgh.pa.us> Modified Files -------------- src/backend/tsearch/to_tsany.c | 23 +++++++++++++++----- src/backend/tsearch/ts_parse.c | 27 ++++++++++++++++++----- src/backend/utils/adt/tsvector.c | 28 ++++++++++++++++++------ src/backend/utils/adt/tsvector_op.c | 40 +++++++++++++++++++++++++++++----- src/include/tsearch/ts_type.h | 43 ++++++++++++++++++++++++++----------- 5 files changed, 126 insertions(+), 35 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Harden tsvector code against overflows. @ 2026-08-10 13:41 Noah Misch <noah@leadboat.com> 0 siblings, 0 replies; 6+ messages in thread From: Noah Misch @ 2026-08-10 13:41 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Harden tsvector code against overflows. The core of this patch is to prevent array_to_tsvector() from generating invalid tsvectors. It did not check for overly-long lexemes (so that WordEntry.len fields could overflow), nor did it check that the total "datalen" fits within MAXSTRPOS (so that WordEntry.pos fields could overflow, and the number of entries in the tsvector could be much more than the normal limit). While the field overflows couldn't do anything much worse than produce a corrupted tsvector value, a sufficiently large number of tsvector entries could cause integer overflows in later processing, such as tsvectorout. Another important fix is to prevent tsvectorrecv() from accepting invalid tsvectors. The main problem there is that it did not reject empty-string lexemes. Hence, even though it did (mostly) enforce the MAXSTRPOS limit, it could still produce a result with an unreasonable number of tsvector entries, if they were primarily empty strings. Also, fix tsvectorout's calculation of its required output buffer size: it was multiplying the string lengths by pg_database_encoding_max_length() for no reason. That contributed to the risk of integer overflow there. With valid tsvector input, there's no risk, but there's still no reason to make the output buffer several times bigger than needed. I also tried to make a couple of related routines more robust, and spent some effort on improving the comments in ts_type.h. Also, standardize on a single spelling of the "string is too long for tsvector" message, using %zu instead of an assortment of formats. These changes aren't security per se but came out of inspecting the code for problems. Reported-by: Yuhang Wu <yuhang@depthfirst.com> and Zhenpeng Lin Reported-by: Zheng Yu <zheng@depthfirst.com> Reported-by: Hcamael <baiyjrh@gmail.com> Author: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Amit Langote <amitlangote09@gmail.com> Backpatch-through: 14 Security: CVE-2026-14662 Branch ------ REL_18_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/e251350573e2ceb846b3b992f483d2d0de7a8a20 Author: Tom Lane <tgl@sss.pgh.pa.us> Modified Files -------------- src/backend/tsearch/to_tsany.c | 23 +++++++++++++++----- src/backend/tsearch/ts_parse.c | 27 ++++++++++++++++++----- src/backend/utils/adt/tsvector.c | 28 ++++++++++++++++++------ src/backend/utils/adt/tsvector_op.c | 40 +++++++++++++++++++++++++++++----- src/include/tsearch/ts_type.h | 43 ++++++++++++++++++++++++++----------- 5 files changed, 126 insertions(+), 35 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Harden tsvector code against overflows. @ 2026-08-10 13:41 Noah Misch <noah@leadboat.com> 0 siblings, 0 replies; 6+ messages in thread From: Noah Misch @ 2026-08-10 13:41 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Harden tsvector code against overflows. The core of this patch is to prevent array_to_tsvector() from generating invalid tsvectors. It did not check for overly-long lexemes (so that WordEntry.len fields could overflow), nor did it check that the total "datalen" fits within MAXSTRPOS (so that WordEntry.pos fields could overflow, and the number of entries in the tsvector could be much more than the normal limit). While the field overflows couldn't do anything much worse than produce a corrupted tsvector value, a sufficiently large number of tsvector entries could cause integer overflows in later processing, such as tsvectorout. Another important fix is to prevent tsvectorrecv() from accepting invalid tsvectors. The main problem there is that it did not reject empty-string lexemes. Hence, even though it did (mostly) enforce the MAXSTRPOS limit, it could still produce a result with an unreasonable number of tsvector entries, if they were primarily empty strings. Also, fix tsvectorout's calculation of its required output buffer size: it was multiplying the string lengths by pg_database_encoding_max_length() for no reason. That contributed to the risk of integer overflow there. With valid tsvector input, there's no risk, but there's still no reason to make the output buffer several times bigger than needed. I also tried to make a couple of related routines more robust, and spent some effort on improving the comments in ts_type.h. Also, standardize on a single spelling of the "string is too long for tsvector" message, using %zu instead of an assortment of formats. These changes aren't security per se but came out of inspecting the code for problems. Reported-by: Yuhang Wu <yuhang@depthfirst.com> and Zhenpeng Lin Reported-by: Zheng Yu <zheng@depthfirst.com> Reported-by: Hcamael <baiyjrh@gmail.com> Author: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Amit Langote <amitlangote09@gmail.com> Backpatch-through: 14 Security: CVE-2026-14662 Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/fe0b5bd6dedc23278f1a48bb3422833e936d881e Author: Tom Lane <tgl@sss.pgh.pa.us> Modified Files -------------- src/backend/tsearch/to_tsany.c | 23 +++++++++++++++----- src/backend/tsearch/ts_parse.c | 27 ++++++++++++++++++----- src/backend/utils/adt/tsvector.c | 28 ++++++++++++++++++------ src/backend/utils/adt/tsvector_op.c | 40 +++++++++++++++++++++++++++++----- src/include/tsearch/ts_type.h | 43 ++++++++++++++++++++++++++----------- 5 files changed, 126 insertions(+), 35 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Harden tsvector code against overflows. @ 2026-08-10 13:41 Noah Misch <noah@leadboat.com> 0 siblings, 0 replies; 6+ messages in thread From: Noah Misch @ 2026-08-10 13:41 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Harden tsvector code against overflows. The core of this patch is to prevent array_to_tsvector() from generating invalid tsvectors. It did not check for overly-long lexemes (so that WordEntry.len fields could overflow), nor did it check that the total "datalen" fits within MAXSTRPOS (so that WordEntry.pos fields could overflow, and the number of entries in the tsvector could be much more than the normal limit). While the field overflows couldn't do anything much worse than produce a corrupted tsvector value, a sufficiently large number of tsvector entries could cause integer overflows in later processing, such as tsvectorout. Another important fix is to prevent tsvectorrecv() from accepting invalid tsvectors. The main problem there is that it did not reject empty-string lexemes. Hence, even though it did (mostly) enforce the MAXSTRPOS limit, it could still produce a result with an unreasonable number of tsvector entries, if they were primarily empty strings. Also, fix tsvectorout's calculation of its required output buffer size: it was multiplying the string lengths by pg_database_encoding_max_length() for no reason. That contributed to the risk of integer overflow there. With valid tsvector input, there's no risk, but there's still no reason to make the output buffer several times bigger than needed. I also tried to make a couple of related routines more robust, and spent some effort on improving the comments in ts_type.h. Also, standardize on a single spelling of the "string is too long for tsvector" message, using %zu instead of an assortment of formats. These changes aren't security per se but came out of inspecting the code for problems. Reported-by: Yuhang Wu <yuhang@depthfirst.com> and Zhenpeng Lin Reported-by: Zheng Yu <zheng@depthfirst.com> Reported-by: Hcamael <baiyjrh@gmail.com> Author: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Amit Langote <amitlangote09@gmail.com> Backpatch-through: 14 Security: CVE-2026-14662 Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/7c1a8805a3ab9a8827621f833cbc4415adfcb9ee Author: Tom Lane <tgl@sss.pgh.pa.us> Modified Files -------------- src/backend/tsearch/to_tsany.c | 23 +++++++++++++++----- src/backend/tsearch/ts_parse.c | 27 ++++++++++++++++++----- src/backend/utils/adt/tsvector.c | 28 ++++++++++++++++++------ src/backend/utils/adt/tsvector_op.c | 40 +++++++++++++++++++++++++++++----- src/include/tsearch/ts_type.h | 43 ++++++++++++++++++++++++++----------- 5 files changed, 126 insertions(+), 35 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Harden tsvector code against overflows. @ 2026-08-10 13:41 Noah Misch <noah@leadboat.com> 0 siblings, 0 replies; 6+ messages in thread From: Noah Misch @ 2026-08-10 13:41 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Harden tsvector code against overflows. The core of this patch is to prevent array_to_tsvector() from generating invalid tsvectors. It did not check for overly-long lexemes (so that WordEntry.len fields could overflow), nor did it check that the total "datalen" fits within MAXSTRPOS (so that WordEntry.pos fields could overflow, and the number of entries in the tsvector could be much more than the normal limit). While the field overflows couldn't do anything much worse than produce a corrupted tsvector value, a sufficiently large number of tsvector entries could cause integer overflows in later processing, such as tsvectorout. Another important fix is to prevent tsvectorrecv() from accepting invalid tsvectors. The main problem there is that it did not reject empty-string lexemes. Hence, even though it did (mostly) enforce the MAXSTRPOS limit, it could still produce a result with an unreasonable number of tsvector entries, if they were primarily empty strings. Also, fix tsvectorout's calculation of its required output buffer size: it was multiplying the string lengths by pg_database_encoding_max_length() for no reason. That contributed to the risk of integer overflow there. With valid tsvector input, there's no risk, but there's still no reason to make the output buffer several times bigger than needed. I also tried to make a couple of related routines more robust, and spent some effort on improving the comments in ts_type.h. Also, standardize on a single spelling of the "string is too long for tsvector" message, using %zu instead of an assortment of formats. These changes aren't security per se but came out of inspecting the code for problems. Reported-by: Yuhang Wu <yuhang@depthfirst.com> and Zhenpeng Lin Reported-by: Zheng Yu <zheng@depthfirst.com> Reported-by: Hcamael <baiyjrh@gmail.com> Author: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Amit Langote <amitlangote09@gmail.com> Backpatch-through: 14 Security: CVE-2026-14662 Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/f443d0a0af03e9edc95f2c6a8da6229d025d0d4e Author: Tom Lane <tgl@sss.pgh.pa.us> Modified Files -------------- src/backend/tsearch/to_tsany.c | 23 +++++++++++++++----- src/backend/tsearch/ts_parse.c | 27 ++++++++++++++++++----- src/backend/utils/adt/tsvector.c | 28 ++++++++++++++++++------ src/backend/utils/adt/tsvector_op.c | 40 +++++++++++++++++++++++++++++----- src/include/tsearch/ts_type.h | 43 ++++++++++++++++++++++++++----------- 5 files changed, 126 insertions(+), 35 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2026-08-10 13:41 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2026-08-10 13:41 pgsql: Harden tsvector code against overflows. Noah Misch <noah@leadboat.com> 2026-08-10 13:41 pgsql: Harden tsvector code against overflows. Noah Misch <noah@leadboat.com> 2026-08-10 13:41 pgsql: Harden tsvector code against overflows. Noah Misch <noah@leadboat.com> 2026-08-10 13:41 pgsql: Harden tsvector code against overflows. Noah Misch <noah@leadboat.com> 2026-08-10 13:41 pgsql: Harden tsvector code against overflows. Noah Misch <noah@leadboat.com> 2026-08-10 13:41 pgsql: Harden tsvector code against overflows. Noah Misch <noah@leadboat.com>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox