pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Invent qsort_interruptible(). 3+ messages / 1 participants [nested] [flat]
* pgsql: Invent qsort_interruptible(). @ 2022-07-12 20:30 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 3+ messages in thread From: Tom Lane @ 2022-07-12 20:30 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Invent qsort_interruptible(). Justin Pryzby reported that some scenarios could cause gathering of extended statistics to spend many seconds in an un-cancelable qsort() operation. To fix, invent qsort_interruptible(), which is just like qsort_arg() except that it will also do CHECK_FOR_INTERRUPTS every so often. This bloats the backend by a couple of kB, which seems like a good investment. (We considered just enabling CHECK_FOR_INTERRUPTS in the existing qsort and qsort_arg functions, but there are some callers for which that'd demonstrably be unsafe. Opt-in seems like a better way.) For now, just apply qsort_interruptible() in statistics collection. There's probably more places where it could be useful, but we can always change other call sites as we find problems. Back-patch to v14. Before that we didn't have extended stats on expressions, so that the problem was less severe. Also, this patch depends on the sort_template infrastructure introduced in v14. Tom Lane and Justin Pryzby Discussion: https://postgr.es/m/20220509000108.GQ28830@telsasoft.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/e64cdab003027acef29e713087fb667e2319f679 Modified Files -------------- src/backend/commands/analyze.c | 25 ++++++++++----------- src/backend/statistics/extended_stats.c | 4 ++-- src/backend/statistics/mcv.c | 14 ++++++------ src/backend/statistics/mvdistinct.c | 4 ++-- src/backend/tsearch/ts_typanalyze.c | 22 ++++++++++--------- src/backend/utils/adt/array_typanalyze.c | 31 ++++++++++++++------------- src/backend/utils/adt/rangetypes_typanalyze.c | 15 +++++++------ src/backend/utils/sort/Makefile | 1 + src/backend/utils/sort/qsort_interruptible.c | 16 ++++++++++++++ src/include/port.h | 3 +++ 10 files changed, 80 insertions(+), 55 deletions(-) ^ permalink raw reply [nested|flat] 3+ messages in thread
* pgsql: Invent qsort_interruptible(). @ 2022-07-12 20:30 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 3+ messages in thread From: Tom Lane @ 2022-07-12 20:30 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Invent qsort_interruptible(). Justin Pryzby reported that some scenarios could cause gathering of extended statistics to spend many seconds in an un-cancelable qsort() operation. To fix, invent qsort_interruptible(), which is just like qsort_arg() except that it will also do CHECK_FOR_INTERRUPTS every so often. This bloats the backend by a couple of kB, which seems like a good investment. (We considered just enabling CHECK_FOR_INTERRUPTS in the existing qsort and qsort_arg functions, but there are some callers for which that'd demonstrably be unsafe. Opt-in seems like a better way.) For now, just apply qsort_interruptible() in statistics collection. There's probably more places where it could be useful, but we can always change other call sites as we find problems. Back-patch to v14. Before that we didn't have extended stats on expressions, so that the problem was less severe. Also, this patch depends on the sort_template infrastructure introduced in v14. Tom Lane and Justin Pryzby Discussion: https://postgr.es/m/20220509000108.GQ28830@telsasoft.com Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/af72b0889441e5ece199f782511d84d72fa6b88c Modified Files -------------- src/backend/commands/analyze.c | 25 ++++++++++----------- src/backend/statistics/extended_stats.c | 4 ++-- src/backend/statistics/mcv.c | 14 ++++++------ src/backend/statistics/mvdistinct.c | 4 ++-- src/backend/tsearch/ts_typanalyze.c | 22 ++++++++++--------- src/backend/utils/adt/array_typanalyze.c | 31 ++++++++++++++------------- src/backend/utils/adt/rangetypes_typanalyze.c | 15 +++++++------ src/backend/utils/sort/Makefile | 1 + src/backend/utils/sort/qsort_interruptible.c | 16 ++++++++++++++ src/include/port.h | 3 +++ 10 files changed, 80 insertions(+), 55 deletions(-) ^ permalink raw reply [nested|flat] 3+ messages in thread
* pgsql: Invent qsort_interruptible(). @ 2022-07-12 20:30 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 3+ messages in thread From: Tom Lane @ 2022-07-12 20:30 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Invent qsort_interruptible(). Justin Pryzby reported that some scenarios could cause gathering of extended statistics to spend many seconds in an un-cancelable qsort() operation. To fix, invent qsort_interruptible(), which is just like qsort_arg() except that it will also do CHECK_FOR_INTERRUPTS every so often. This bloats the backend by a couple of kB, which seems like a good investment. (We considered just enabling CHECK_FOR_INTERRUPTS in the existing qsort and qsort_arg functions, but there are some callers for which that'd demonstrably be unsafe. Opt-in seems like a better way.) For now, just apply qsort_interruptible() in statistics collection. There's probably more places where it could be useful, but we can always change other call sites as we find problems. Back-patch to v14. Before that we didn't have extended stats on expressions, so that the problem was less severe. Also, this patch depends on the sort_template infrastructure introduced in v14. Tom Lane and Justin Pryzby Discussion: https://postgr.es/m/20220509000108.GQ28830@telsasoft.com Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/12c99c88442d18e82daf521907a6e3d9003295ee Modified Files -------------- src/backend/commands/analyze.c | 25 ++++++++++----------- src/backend/statistics/extended_stats.c | 4 ++-- src/backend/statistics/mcv.c | 14 ++++++------ src/backend/statistics/mvdistinct.c | 4 ++-- src/backend/tsearch/ts_typanalyze.c | 22 ++++++++++--------- src/backend/utils/adt/array_typanalyze.c | 31 ++++++++++++++------------- src/backend/utils/adt/rangetypes_typanalyze.c | 15 +++++++------ src/backend/utils/sort/Makefile | 1 + src/backend/utils/sort/qsort_interruptible.c | 16 ++++++++++++++ src/include/port.h | 3 +++ 10 files changed, 80 insertions(+), 55 deletions(-) ^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2022-07-12 20:30 UTC | newest] Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2022-07-12 20:30 pgsql: Invent qsort_interruptible(). Tom Lane <tgl@sss.pgh.pa.us> 2022-07-12 20:30 pgsql: Invent qsort_interruptible(). Tom Lane <tgl@sss.pgh.pa.us> 2022-07-12 20:30 pgsql: Invent qsort_interruptible(). Tom Lane <tgl@sss.pgh.pa.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox