pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
pgsql: Free memory after building each statistics object
6+ messages / 1 participants
[nested] [flat]

* pgsql: Free memory after building each statistics object
@ 2021-09-23 16:20  Tomas Vondra <tomas.vondra@postgresql.org>
  0 siblings, 0 replies; 6+ messages in thread

From: Tomas Vondra @ 2021-09-23 16:20 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Free memory after building each statistics object

Until now, all extended statistics on a given relation were built in the
same memory context, without resetting. Some of the memory was released
explicitly, but not all of it - for example memory allocated while
detoasting values is hard to free. This is how it worked since extended
statistics were introduced in PostgreSQL 10, but adding support for
extended stats on expressions made the issue somewhat worse as it
increases the number of statistics to build.

Fixed by adding a memory context which gets reset after building each
statistics object (all the statistics kinds included in it). Resetting
it after building each statistics kind would be even better, but it
would require more invasive changes and copying of results, making it
harder to backpatch.

Backpatch to PostgreSQL 10, where extended statistics were introduced.

Author: Justin Pryzby
Reported-by: Justin Pryzby
Reviewed-by: Tomas Vondra
Backpatch-through: 10
Discussion: https://www.postgresql.org/message-id/20210915200928.GP831%40telsasoft.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/83772cc78e0392a247231ba510c61b6612b93b3f

Modified Files
--------------
src/backend/statistics/extended_stats.c | 18 +++++++++++-------
1 file changed, 11 insertions(+), 7 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: Free memory after building each statistics object
@ 2021-09-23 16:29  Tomas Vondra <tomas.vondra@postgresql.org>
  0 siblings, 0 replies; 6+ messages in thread

From: Tomas Vondra @ 2021-09-23 16:29 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Free memory after building each statistics object

Until now, all extended statistics on a given relation were built in the
same memory context, without resetting. Some of the memory was released
explicitly, but not all of it - for example memory allocated while
detoasting values is hard to free. This is how it worked since extended
statistics were introduced in PostgreSQL 10, but adding support for
extended stats on expressions made the issue somewhat worse as it
increases the number of statistics to build.

Fixed by adding a memory context which gets reset after building each
statistics object (all the statistics kinds included in it). Resetting
it after building each statistics kind would be even better, but it
would require more invasive changes and copying of results, making it
harder to backpatch.

Backpatch to PostgreSQL 10, where extended statistics were introduced.

Author: Justin Pryzby
Reported-by: Justin Pryzby
Reviewed-by: Tomas Vondra
Backpatch-through: 10
Discussion: https://www.postgresql.org/message-id/20210915200928.GP831%40telsasoft.com

Branch
------
REL_14_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/bb7628e55eda6f450f0f824d11c9a34b11f6bb12

Modified Files
--------------
src/backend/statistics/extended_stats.c | 18 +++++++++++-------
1 file changed, 11 insertions(+), 7 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: Free memory after building each statistics object
@ 2021-09-23 16:40  Tomas Vondra <tomas.vondra@postgresql.org>
  0 siblings, 0 replies; 6+ messages in thread

From: Tomas Vondra @ 2021-09-23 16:40 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Free memory after building each statistics object

Until now, all extended statistics on a given relation were built in the
same memory context, without resetting. Some of the memory was released
explicitly, but not all of it - for example memory allocated while
detoasting values is hard to free. This is how it worked since extended
statistics were introduced in PostgreSQL 10, but adding support for
extended stats on expressions made the issue somewhat worse as it
increases the number of statistics to build.

Fixed by adding a memory context which gets reset after building each
statistics object (all the statistics kinds included in it). Resetting
it after building each statistics kind would be even better, but it
would require more invasive changes and copying of results, making it
harder to backpatch.

Backpatch to PostgreSQL 10, where extended statistics were introduced.

Author: Justin Pryzby
Reported-by: Justin Pryzby
Reviewed-by: Tomas Vondra
Backpatch-through: 10
Discussion: https://www.postgresql.org/message-id/20210915200928.GP831%40telsasoft.com

Branch
------
REL_13_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/b564eb0181e678ee58c1be2e9f3c53eeb68f8c4b

Modified Files
--------------
src/backend/statistics/extended_stats.c | 16 +++++++++++-----
1 file changed, 11 insertions(+), 5 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: Free memory after building each statistics object
@ 2021-09-23 16:47  Tomas Vondra <tomas.vondra@postgresql.org>
  0 siblings, 0 replies; 6+ messages in thread

From: Tomas Vondra @ 2021-09-23 16:47 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Free memory after building each statistics object

Until now, all extended statistics on a given relation were built in the
same memory context, without resetting. Some of the memory was released
explicitly, but not all of it - for example memory allocated while
detoasting values is hard to free. This is how it worked since extended
statistics were introduced in PostgreSQL 10, but adding support for
extended stats on expressions made the issue somewhat worse as it
increases the number of statistics to build.

Fixed by adding a memory context which gets reset after building each
statistics object (all the statistics kinds included in it). Resetting
it after building each statistics kind would be even better, but it
would require more invasive changes and copying of results, making it
harder to backpatch.

Backpatch to PostgreSQL 10, where extended statistics were introduced.

Author: Justin Pryzby
Reported-by: Justin Pryzby
Reviewed-by: Tomas Vondra
Backpatch-through: 10
Discussion: https://www.postgresql.org/message-id/20210915200928.GP831%40telsasoft.com

Branch
------
REL_12_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/16d394c05065d93b2ceca5b9be8dfc608193043a

Modified Files
--------------
src/backend/statistics/extended_stats.c | 16 +++++++++++-----
1 file changed, 11 insertions(+), 5 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: Free memory after building each statistics object
@ 2021-09-23 16:53  Tomas Vondra <tomas.vondra@postgresql.org>
  0 siblings, 0 replies; 6+ messages in thread

From: Tomas Vondra @ 2021-09-23 16:53 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Free memory after building each statistics object

Until now, all extended statistics on a given relation were built in the
same memory context, without resetting. Some of the memory was released
explicitly, but not all of it - for example memory allocated while
detoasting values is hard to free. This is how it worked since extended
statistics were introduced in PostgreSQL 10, but adding support for
extended stats on expressions made the issue somewhat worse as it
increases the number of statistics to build.

Fixed by adding a memory context which gets reset after building each
statistics object (all the statistics kinds included in it). Resetting
it after building each statistics kind would be even better, but it
would require more invasive changes and copying of results, making it
harder to backpatch.

Backpatch to PostgreSQL 10, where extended statistics were introduced.

Author: Justin Pryzby
Reported-by: Justin Pryzby
Reviewed-by: Tomas Vondra
Backpatch-through: 10
Discussion: https://www.postgresql.org/message-id/20210915200928.GP831%40telsasoft.com

Branch
------
REL_11_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/ac7290a20ec52fa95316e30191827594b60470d3

Modified Files
--------------
src/backend/statistics/extended_stats.c | 16 +++++++++++-----
1 file changed, 11 insertions(+), 5 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: Free memory after building each statistics object
@ 2021-09-23 17:01  Tomas Vondra <tomas.vondra@postgresql.org>
  0 siblings, 0 replies; 6+ messages in thread

From: Tomas Vondra @ 2021-09-23 17:01 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Free memory after building each statistics object

Until now, all extended statistics on a given relation were built in the
same memory context, without resetting. Some of the memory was released
explicitly, but not all of it - for example memory allocated while
detoasting values is hard to free. This is how it worked since extended
statistics were introduced in PostgreSQL 10, but adding support for
extended stats on expressions made the issue somewhat worse as it
increases the number of statistics to build.

Fixed by adding a memory context which gets reset after building each
statistics object (all the statistics kinds included in it). Resetting
it after building each statistics kind would be even better, but it
would require more invasive changes and copying of results, making it
harder to backpatch.

Backpatch to PostgreSQL 10, where extended statistics were introduced.

Author: Justin Pryzby
Reported-by: Justin Pryzby
Reviewed-by: Tomas Vondra
Backpatch-through: 10
Discussion: https://www.postgresql.org/message-id/20210915200928.GP831%40telsasoft.com

Branch
------
REL_10_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/3aac99068cd7abe9d6fbc8c0faa361d75f7d0bfc

Modified Files
--------------
src/backend/statistics/extended_stats.c | 19 +++++++++++++------
1 file changed, 13 insertions(+), 6 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread


end of thread, other threads:[~2021-09-23 17:01 UTC | newest]

Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-09-23 16:20 pgsql: Free memory after building each statistics object Tomas Vondra <tomas.vondra@postgresql.org>
2021-09-23 16:29 pgsql: Free memory after building each statistics object Tomas Vondra <tomas.vondra@postgresql.org>
2021-09-23 16:40 pgsql: Free memory after building each statistics object Tomas Vondra <tomas.vondra@postgresql.org>
2021-09-23 16:47 pgsql: Free memory after building each statistics object Tomas Vondra <tomas.vondra@postgresql.org>
2021-09-23 16:53 pgsql: Free memory after building each statistics object Tomas Vondra <tomas.vondra@postgresql.org>
2021-09-23 17:01 pgsql: Free memory after building each statistics object Tomas Vondra <tomas.vondra@postgresql.org>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox