public inbox for [email protected]help / color / mirror / Atom feed
[PATCH v1] Fine tune documentation for tableam 113+ messages / 2 participants [nested] [flat]
* [PATCH v1] Fine tune documentation for tableam @ 2019-04-04 05:48 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Justin Pryzby @ 2019-04-04 05:48 UTC (permalink / raw) Added at commit b73c3a11963c8bb783993cfffabb09f558f86e37 --- doc/src/sgml/catalogs.sgml | 2 +- doc/src/sgml/config.sgml | 4 ++-- doc/src/sgml/ref/select_into.sgml | 6 +++--- doc/src/sgml/storage.sgml | 17 ++++++++------- doc/src/sgml/tableam.sgml | 44 ++++++++++++++++++++------------------- 5 files changed, 38 insertions(+), 35 deletions(-) diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml index 58c8c96..40ddec4 100644 --- a/doc/src/sgml/catalogs.sgml +++ b/doc/src/sgml/catalogs.sgml @@ -587,7 +587,7 @@ The catalog <structname>pg_am</structname> stores information about relation access methods. There is one row for each access method supported by the system. - Currently, only table and indexes have access methods. The requirements for table + Currently, only tables and indexes have access methods. The requirements for table and index access methods are discussed in detail in <xref linkend="tableam"/> and <xref linkend="indexam"/> respectively. </para> diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index 4a9a1e8..90b478d 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -7306,8 +7306,8 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv; This parameter specifies the default table access method to use when creating tables or materialized views if the <command>CREATE</command> command does not explicitly specify an access method, or when - <command>SELECT ... INTO</command> is used, which does not allow to - specify a table access method. The default is <literal>heap</literal>. + <command>SELECT ... INTO</command> is used, which does not allow + specification of a table access method. The default is <literal>heap</literal>. </para> </listitem> </varlistentry> diff --git a/doc/src/sgml/ref/select_into.sgml b/doc/src/sgml/ref/select_into.sgml index 17bed24..1443d79 100644 --- a/doc/src/sgml/ref/select_into.sgml +++ b/doc/src/sgml/ref/select_into.sgml @@ -106,11 +106,11 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac </para> <para> - In contrast to <command>CREATE TABLE AS</command> <command>SELECT - INTO</command> does not allow to specify properties like a table's access + In contrast to <command>CREATE TABLE AS</command>, <command>SELECT + INTO</command> does not allow specification of properties like a table's access method with <xref linkend="sql-createtable-method" /> or the table's tablespace with <xref linkend="sql-createtable-tablespace" />. Use <xref - linkend="sql-createtableas"/> if necessary. Therefore the default table + linkend="sql-createtableas"/> if necessary. Therefore, the default table access method is chosen for the new table. See <xref linkend="guc-default-table-access-method"/> for more information. </para> diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml index 62333e3..5dfca1b 100644 --- a/doc/src/sgml/storage.sgml +++ b/doc/src/sgml/storage.sgml @@ -189,11 +189,11 @@ there. </para> <para> - Note that the following sections describe the way the builtin + Note that the following sections describe the behavior of the builtin <literal>heap</literal> <link linkend="tableam">table access method</link>, - and the builtin <link linkend="indexam">index access methods</link> work. Due - to the extensible nature of <productname>PostgreSQL</productname> other types - of access method might work similar or not. + and the builtin <link linkend="indexam">index access methods</link>. Due + to the extensible nature of <productname>PostgreSQL</productname>, other + access methods might work differently. </para> <para> @@ -703,11 +703,12 @@ erased (they will be recreated automatically as needed). This section provides an overview of the page format used within <productname>PostgreSQL</productname> tables and indexes.<footnote> <para> - Actually, neither table nor index access methods need not use this page - format. All the existing index methods do use this basic format, but the + Actually, use of this page format is not required for either table or index + access methods. + The <literal>heap</literal> table access method always uses this format. + All the existing index methods also use the basic format, but the data kept on index metapages usually doesn't follow the item layout - rules. The <literal>heap</literal> table access method also always uses - this format. + rules. </para> </footnote> Sequences and <acronym>TOAST</acronym> tables are formatted just like a regular table. diff --git a/doc/src/sgml/tableam.sgml b/doc/src/sgml/tableam.sgml index 8d9bfd8..0a89935 100644 --- a/doc/src/sgml/tableam.sgml +++ b/doc/src/sgml/tableam.sgml @@ -48,54 +48,56 @@ callbacks and their behavior is defined in the <structname>TableAmRoutine</structname> structure (with comments inside the struct defining the requirements for callbacks). Most callbacks have - wrapper functions, which are documented for the point of view of a user, - rather than an implementor, of the table access method. For details, + wrapper functions, which are documented from the point of view of a user + (rather than an implementor) of the table access method. For details, please refer to the <ulink url="https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/include/access/tableam.h;hb=HEAD...; <filename>src/include/access/tableam.h</filename></ulink> file. </para> <para> - To implement a access method, an implementor will typically need to - implement a AM specific type of tuple table slot (see + To implement an access method, an implementor will typically need to + implement an AM-specific type of tuple table slot (see <ulink url="https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/include/executor/tuptable.h;hb=HEAD...; - <filename>src/include/executor/tuptable.h</filename></ulink>) which allows + <filename>src/include/executor/tuptable.h</filename></ulink>), which allows code outside the access method to hold references to tuples of the AM, and to access the columns of the tuple. </para> <para> - Currently the the way an AM actually stores data is fairly - unconstrained. It is e.g. possible to use postgres' shared buffer cache, - but not required. In case shared buffers are used, it likely makes to - postgres' standard page layout described in <xref + Currently, the way an AM actually stores data is fairly + unconstrained. For example, it's possible but not required to use postgres' + shared buffer cache. In case it is used, it likely makes +XXX something missing here ? + to postgres' standard page layout described in <xref linkend="storage-page-layout"/>. </para> <para> One fairly large constraint of the table access method API is that, currently, if the AM wants to support modifications and/or indexes, it is - necessary that each tuple has a tuple identifier (<acronym>TID</acronym>) + necessary for each tuple to have a tuple identifier (<acronym>TID</acronym>) consisting of a block number and an item number (see also <xref linkend="storage-page-layout"/>). It is not strictly necessary that the - sub-parts of <acronym>TIDs</acronym> have the same meaning they e.g. have + sub-parts of <acronym>TIDs</acronym> have the same meaning as used for <literal>heap</literal>, but if bitmap scan support is desired (it is optional), the block number needs to provide locality. </para> <para> - For crash safety an AM can use postgres' <link - linkend="wal"><acronym>WAL</acronym></link>, or a custom approach can be - implemented. If <acronym>WAL</acronym> is chosen, either <link - linkend="generic-wal">Generic WAL Records</link> can be used — which - implies higher WAL volume but is easy, or a new type of - <acronym>WAL</acronym> records can be implemented — but that - currently requires modifications of core code (namely modifying + For crash safety, an AM can use postgres' <link + linkend="wal"><acronym>WAL</acronym></link>, or a custom implementation. + If <acronym>WAL</acronym> is chosen, either <link + linkend="generic-wal">Generic WAL Records</link> can be used, + or a new type of <acronym>WAL</acronym> records can be implemented. + Generic WAL Records are easy, but imply higher WAL volume. + Implementation of a new type of WAL record + currently requires modifications to core code (specifically, <filename>src/include/access/rmgrlist.h</filename>). </para> <para> To implement transactional support in a manner that allows different table - access methods be accessed within a single transaction, it likely is + access methods be accessed within a single transaction, it's likely necessary to closely integrate with the machinery in <filename>src/backend/access/transam/xlog.c</filename>. </para> @@ -103,8 +105,8 @@ <para> Any developer of a new <literal>table access method</literal> can refer to the existing <literal>heap</literal> implementation present in - <filename>src/backend/heap/heapam_handler.c</filename> for more details of - how it is implemented. + <filename>src/backend/heap/heapam_handler.c</filename> for details of + its implementation. </para> </chapter> -- 2.1.4 --ibTvN161/egqYuK8-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v1] Fine tune documentation for tableam @ 2019-04-04 05:48 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Justin Pryzby @ 2019-04-04 05:48 UTC (permalink / raw) Added at commit b73c3a11963c8bb783993cfffabb09f558f86e37 --- doc/src/sgml/catalogs.sgml | 2 +- doc/src/sgml/config.sgml | 4 ++-- doc/src/sgml/ref/select_into.sgml | 6 +++--- doc/src/sgml/storage.sgml | 17 ++++++++------- doc/src/sgml/tableam.sgml | 44 ++++++++++++++++++++------------------- 5 files changed, 38 insertions(+), 35 deletions(-) diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml index 58c8c96..40ddec4 100644 --- a/doc/src/sgml/catalogs.sgml +++ b/doc/src/sgml/catalogs.sgml @@ -587,7 +587,7 @@ The catalog <structname>pg_am</structname> stores information about relation access methods. There is one row for each access method supported by the system. - Currently, only table and indexes have access methods. The requirements for table + Currently, only tables and indexes have access methods. The requirements for table and index access methods are discussed in detail in <xref linkend="tableam"/> and <xref linkend="indexam"/> respectively. </para> diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index 4a9a1e8..90b478d 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -7306,8 +7306,8 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv; This parameter specifies the default table access method to use when creating tables or materialized views if the <command>CREATE</command> command does not explicitly specify an access method, or when - <command>SELECT ... INTO</command> is used, which does not allow to - specify a table access method. The default is <literal>heap</literal>. + <command>SELECT ... INTO</command> is used, which does not allow + specification of a table access method. The default is <literal>heap</literal>. </para> </listitem> </varlistentry> diff --git a/doc/src/sgml/ref/select_into.sgml b/doc/src/sgml/ref/select_into.sgml index 17bed24..1443d79 100644 --- a/doc/src/sgml/ref/select_into.sgml +++ b/doc/src/sgml/ref/select_into.sgml @@ -106,11 +106,11 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac </para> <para> - In contrast to <command>CREATE TABLE AS</command> <command>SELECT - INTO</command> does not allow to specify properties like a table's access + In contrast to <command>CREATE TABLE AS</command>, <command>SELECT + INTO</command> does not allow specification of properties like a table's access method with <xref linkend="sql-createtable-method" /> or the table's tablespace with <xref linkend="sql-createtable-tablespace" />. Use <xref - linkend="sql-createtableas"/> if necessary. Therefore the default table + linkend="sql-createtableas"/> if necessary. Therefore, the default table access method is chosen for the new table. See <xref linkend="guc-default-table-access-method"/> for more information. </para> diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml index 62333e3..5dfca1b 100644 --- a/doc/src/sgml/storage.sgml +++ b/doc/src/sgml/storage.sgml @@ -189,11 +189,11 @@ there. </para> <para> - Note that the following sections describe the way the builtin + Note that the following sections describe the behavior of the builtin <literal>heap</literal> <link linkend="tableam">table access method</link>, - and the builtin <link linkend="indexam">index access methods</link> work. Due - to the extensible nature of <productname>PostgreSQL</productname> other types - of access method might work similar or not. + and the builtin <link linkend="indexam">index access methods</link>. Due + to the extensible nature of <productname>PostgreSQL</productname>, other + access methods might work differently. </para> <para> @@ -703,11 +703,12 @@ erased (they will be recreated automatically as needed). This section provides an overview of the page format used within <productname>PostgreSQL</productname> tables and indexes.<footnote> <para> - Actually, neither table nor index access methods need not use this page - format. All the existing index methods do use this basic format, but the + Actually, use of this page format is not required for either table or index + access methods. + The <literal>heap</literal> table access method always uses this format. + All the existing index methods also use the basic format, but the data kept on index metapages usually doesn't follow the item layout - rules. The <literal>heap</literal> table access method also always uses - this format. + rules. </para> </footnote> Sequences and <acronym>TOAST</acronym> tables are formatted just like a regular table. diff --git a/doc/src/sgml/tableam.sgml b/doc/src/sgml/tableam.sgml index 8d9bfd8..0a89935 100644 --- a/doc/src/sgml/tableam.sgml +++ b/doc/src/sgml/tableam.sgml @@ -48,54 +48,56 @@ callbacks and their behavior is defined in the <structname>TableAmRoutine</structname> structure (with comments inside the struct defining the requirements for callbacks). Most callbacks have - wrapper functions, which are documented for the point of view of a user, - rather than an implementor, of the table access method. For details, + wrapper functions, which are documented from the point of view of a user + (rather than an implementor) of the table access method. For details, please refer to the <ulink url="https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/include/access/tableam.h;hb=HEAD...; <filename>src/include/access/tableam.h</filename></ulink> file. </para> <para> - To implement a access method, an implementor will typically need to - implement a AM specific type of tuple table slot (see + To implement an access method, an implementor will typically need to + implement an AM-specific type of tuple table slot (see <ulink url="https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/include/executor/tuptable.h;hb=HEAD...; - <filename>src/include/executor/tuptable.h</filename></ulink>) which allows + <filename>src/include/executor/tuptable.h</filename></ulink>), which allows code outside the access method to hold references to tuples of the AM, and to access the columns of the tuple. </para> <para> - Currently the the way an AM actually stores data is fairly - unconstrained. It is e.g. possible to use postgres' shared buffer cache, - but not required. In case shared buffers are used, it likely makes to - postgres' standard page layout described in <xref + Currently, the way an AM actually stores data is fairly + unconstrained. For example, it's possible but not required to use postgres' + shared buffer cache. In case it is used, it likely makes +XXX something missing here ? + to postgres' standard page layout described in <xref linkend="storage-page-layout"/>. </para> <para> One fairly large constraint of the table access method API is that, currently, if the AM wants to support modifications and/or indexes, it is - necessary that each tuple has a tuple identifier (<acronym>TID</acronym>) + necessary for each tuple to have a tuple identifier (<acronym>TID</acronym>) consisting of a block number and an item number (see also <xref linkend="storage-page-layout"/>). It is not strictly necessary that the - sub-parts of <acronym>TIDs</acronym> have the same meaning they e.g. have + sub-parts of <acronym>TIDs</acronym> have the same meaning as used for <literal>heap</literal>, but if bitmap scan support is desired (it is optional), the block number needs to provide locality. </para> <para> - For crash safety an AM can use postgres' <link - linkend="wal"><acronym>WAL</acronym></link>, or a custom approach can be - implemented. If <acronym>WAL</acronym> is chosen, either <link - linkend="generic-wal">Generic WAL Records</link> can be used — which - implies higher WAL volume but is easy, or a new type of - <acronym>WAL</acronym> records can be implemented — but that - currently requires modifications of core code (namely modifying + For crash safety, an AM can use postgres' <link + linkend="wal"><acronym>WAL</acronym></link>, or a custom implementation. + If <acronym>WAL</acronym> is chosen, either <link + linkend="generic-wal">Generic WAL Records</link> can be used, + or a new type of <acronym>WAL</acronym> records can be implemented. + Generic WAL Records are easy, but imply higher WAL volume. + Implementation of a new type of WAL record + currently requires modifications to core code (specifically, <filename>src/include/access/rmgrlist.h</filename>). </para> <para> To implement transactional support in a manner that allows different table - access methods be accessed within a single transaction, it likely is + access methods be accessed within a single transaction, it's likely necessary to closely integrate with the machinery in <filename>src/backend/access/transam/xlog.c</filename>. </para> @@ -103,8 +105,8 @@ <para> Any developer of a new <literal>table access method</literal> can refer to the existing <literal>heap</literal> implementation present in - <filename>src/backend/heap/heapam_handler.c</filename> for more details of - how it is implemented. + <filename>src/backend/heap/heapam_handler.c</filename> for details of + its implementation. </para> </chapter> -- 2.1.4 --ibTvN161/egqYuK8 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v1-0001-Fine-tune-documentation-for-tableam.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v4 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1212 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 803 insertions(+), 1052 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..34896b303ac 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all objects the same, but to register a +new kind of an object with it, you need to fill in a few callback +functions, see ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..99240679e09 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ - + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, true); + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +841,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +892,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------7349B9D66ED595EF3B1030B4-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v3 1/1] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 43 +- src/backend/storage/buffer/localbuf.c | 2 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 98 ++- src/backend/utils/cache/plancache.c | 50 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/cache/syscache.c | 2 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1139 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 15 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 21 files changed, 795 insertions(+), 1020 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c46b8abad12..8b77fb68941 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -739,7 +751,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); isExtend = (blockNum == P_NEW); @@ -1858,7 +1870,7 @@ BufferSync(int flags) WritebackContext wb_context; /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Unless this is a shutdown checkpoint or we have been explicitly told, @@ -2335,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context) */ /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); num_to_scan = bufs_to_lap; num_written = 0; @@ -3488,7 +3500,7 @@ FlushRelationBuffers(Relation rel) } /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3561,7 +3573,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3640,7 +3652,7 @@ FlushDatabaseBuffers(Oid dbid) BufferDesc *bufHdr; /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3738,7 +3750,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4802,3 +4814,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 04b3558ea33..811e0e3a45e 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index 931ed679307..5d2394e6780 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -96,7 +96,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -339,6 +339,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1429,7 +1447,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1611,7 +1629,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1741,7 +1759,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1779,7 +1797,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3635,3 +3653,19 @@ data_sync_elevel(int elevel) { return data_sync_retry ? elevel : PANIC; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index fa2b49c676e..24751c0d3f4 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,66 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* support for catcache refcount management */ +static inline void +ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs); +} +static inline void +ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs); +} + +static inline void +ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs); +} + +static inline void +ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs); +} + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1331,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1432,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1644,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1607,7 +1668,7 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ctlist = NIL; @@ -2062,14 +2123,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) { + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) +{ + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2083,9 +2149,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) +{ + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) { + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..484f14f5524 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +static inline void +ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} +static inline void +ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1254,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1417,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1476,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2230,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index e4dc4ee34ee..d6fe6a5bd5a 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -994,7 +994,7 @@ static const struct cachedesc cacheinfo[] = { } }; -static CatCache *SysCache[SysCacheSize]; + CatCache *SysCache[SysCacheSize]; static bool CacheInitialized = false; diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..890db7d1e66 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context references. +ResourceOwner treates all objects the same, but to register a new kind of +an object with it, you need to fill in a few callback functions, see +ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callabacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..c49758cb20d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) - -/* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. + * ResourceElem represents a reference associated with a resource owner. * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. - * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,45 +153,34 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ - -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* @@ -207,205 +189,227 @@ ResourceArrayInit(ResourceArray *resarr, Datum invalidval) * This is separate from actually inserting a resource because if we run out * of memory, it's critical to do so *before* acquiring the resource. */ -static void -ResourceArrayEnlarge(ResourceArray *resarr) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) + if (owner->narr < RESOWNER_ARRAY_SIZE) return; /* no work needed */ - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; - - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; - - if (olditemsarr != NULL) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. - */ - for (i = 0; i < oldcap; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) + { + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); + } + + /* And release old hash table. */ + pfree(oldhash); } + } - /* And release old array. */ - pfree(olditemsarr); + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - Assert(resarr->nitems < resarr->maxitems); + Assert(owner->nhash < owner->grow_at); } /* - * Add a resource to ResourceArray + * Remember that an object is owner by a ReourceOwner * - * Caller must have previously done ResourceArrayEnlarge() + * Caller must have previously done ResourceOwnerEnlarge() */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 idx; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Append to linear array. */ - idx = resarr->nitems; - } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Remove a resource from ResourceArray + * Forget that an object is owned by a ResourceOwner * - * Returns true on success, false if resource was not found. + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. * - * Note: if same resource ID appears more than once, one instance is removed. + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 i, - idx, - lastidx = resarr->lastidx; + idx; - Assert(value != resarr->invalidval); +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) - { - if (resarr->itemsarr[i] == value) - { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + +#ifdef RESOWNER_STATS + narray_lookups++; +#endif + + return; } } - else + + /* Search hash */ + if (owner->nhash > 0) { - uint32 mask = resarr->capacity - 1; + uint32 mask = owner->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) + for (i = 0; i < owner->capacity; i++) { - if (resarr->itemsarr[idx] == value) + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; } idx = (idx + 1) & mask; } } - return false; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Call the ReleaseResource callback on entries with given 'phase'. */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +static void +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - if (resarr->nitems == 0) - return false; + bool found; + int capacity; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else + /* First handle all the entries in the array. */ + do { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; - - for (;;) + found = false; + for (int i = 0; i < owner->narr; i++) { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } } - } - *value = resarr->itemsarr[resarr->lastidx]; - return true; -} + /* + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. + */ + } while (found && owner->narr > 0); -/* - * Trash a ResourceArray (we don't care about its state after this) - */ -static void -ResourceArrayFree(ResourceArray *resarr) -{ - if (resarr->itemsarr) - pfree(resarr->itemsarr); + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) + { + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } + } + + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } @@ -437,17 +441,10 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name) parent->firstchild = owner; } - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif return owner; } @@ -486,6 +483,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +503,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +518,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +533,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +577,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +600,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - ReleaseCachedPlan(res, true); + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) + { + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + CurrentResourceOwner = save; } @@ -730,19 +652,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +676,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +830,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +881,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..1ed1580c644 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..281f85586fa 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,21 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +static inline void +ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} +static inline void +ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------3CC8131494C3F8E34BADAF02-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v3 1/1] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 43 +- src/backend/storage/buffer/localbuf.c | 2 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 98 ++- src/backend/utils/cache/plancache.c | 50 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/cache/syscache.c | 2 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1139 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 15 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 21 files changed, 795 insertions(+), 1020 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c46b8abad12..8b77fb68941 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -739,7 +751,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); isExtend = (blockNum == P_NEW); @@ -1858,7 +1870,7 @@ BufferSync(int flags) WritebackContext wb_context; /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Unless this is a shutdown checkpoint or we have been explicitly told, @@ -2335,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context) */ /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); num_to_scan = bufs_to_lap; num_written = 0; @@ -3488,7 +3500,7 @@ FlushRelationBuffers(Relation rel) } /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3561,7 +3573,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3640,7 +3652,7 @@ FlushDatabaseBuffers(Oid dbid) BufferDesc *bufHdr; /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3738,7 +3750,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4802,3 +4814,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 04b3558ea33..811e0e3a45e 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index 931ed679307..5d2394e6780 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -96,7 +96,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -339,6 +339,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1429,7 +1447,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1611,7 +1629,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1741,7 +1759,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1779,7 +1797,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3635,3 +3653,19 @@ data_sync_elevel(int elevel) { return data_sync_retry ? elevel : PANIC; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index fa2b49c676e..24751c0d3f4 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,66 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* support for catcache refcount management */ +static inline void +ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs); +} +static inline void +ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs); +} + +static inline void +ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs); +} + +static inline void +ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs); +} + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1331,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1432,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1644,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1607,7 +1668,7 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ctlist = NIL; @@ -2062,14 +2123,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) { + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) +{ + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2083,9 +2149,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) +{ + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) { + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..484f14f5524 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +static inline void +ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} +static inline void +ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1254,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1417,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1476,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2230,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index e4dc4ee34ee..d6fe6a5bd5a 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -994,7 +994,7 @@ static const struct cachedesc cacheinfo[] = { } }; -static CatCache *SysCache[SysCacheSize]; + CatCache *SysCache[SysCacheSize]; static bool CacheInitialized = false; diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..890db7d1e66 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context references. +ResourceOwner treates all objects the same, but to register a new kind of +an object with it, you need to fill in a few callback functions, see +ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callabacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..c49758cb20d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) - -/* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. + * ResourceElem represents a reference associated with a resource owner. * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. - * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,45 +153,34 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ - -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* @@ -207,205 +189,227 @@ ResourceArrayInit(ResourceArray *resarr, Datum invalidval) * This is separate from actually inserting a resource because if we run out * of memory, it's critical to do so *before* acquiring the resource. */ -static void -ResourceArrayEnlarge(ResourceArray *resarr) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) + if (owner->narr < RESOWNER_ARRAY_SIZE) return; /* no work needed */ - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; - - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; - - if (olditemsarr != NULL) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. - */ - for (i = 0; i < oldcap; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) + { + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); + } + + /* And release old hash table. */ + pfree(oldhash); } + } - /* And release old array. */ - pfree(olditemsarr); + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - Assert(resarr->nitems < resarr->maxitems); + Assert(owner->nhash < owner->grow_at); } /* - * Add a resource to ResourceArray + * Remember that an object is owner by a ReourceOwner * - * Caller must have previously done ResourceArrayEnlarge() + * Caller must have previously done ResourceOwnerEnlarge() */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 idx; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Append to linear array. */ - idx = resarr->nitems; - } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Remove a resource from ResourceArray + * Forget that an object is owned by a ResourceOwner * - * Returns true on success, false if resource was not found. + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. * - * Note: if same resource ID appears more than once, one instance is removed. + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 i, - idx, - lastidx = resarr->lastidx; + idx; - Assert(value != resarr->invalidval); +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) - { - if (resarr->itemsarr[i] == value) - { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + +#ifdef RESOWNER_STATS + narray_lookups++; +#endif + + return; } } - else + + /* Search hash */ + if (owner->nhash > 0) { - uint32 mask = resarr->capacity - 1; + uint32 mask = owner->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) + for (i = 0; i < owner->capacity; i++) { - if (resarr->itemsarr[idx] == value) + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; } idx = (idx + 1) & mask; } } - return false; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Call the ReleaseResource callback on entries with given 'phase'. */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +static void +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - if (resarr->nitems == 0) - return false; + bool found; + int capacity; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else + /* First handle all the entries in the array. */ + do { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; - - for (;;) + found = false; + for (int i = 0; i < owner->narr; i++) { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } } - } - *value = resarr->itemsarr[resarr->lastidx]; - return true; -} + /* + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. + */ + } while (found && owner->narr > 0); -/* - * Trash a ResourceArray (we don't care about its state after this) - */ -static void -ResourceArrayFree(ResourceArray *resarr) -{ - if (resarr->itemsarr) - pfree(resarr->itemsarr); + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) + { + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } + } + + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } @@ -437,17 +441,10 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name) parent->firstchild = owner; } - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif return owner; } @@ -486,6 +483,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +503,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +518,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +533,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +577,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +600,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - ReleaseCachedPlan(res, true); + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) + { + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + CurrentResourceOwner = save; } @@ -730,19 +652,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +676,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +830,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +881,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..1ed1580c644 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..281f85586fa 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,21 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +static inline void +ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} +static inline void +ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------3CC8131494C3F8E34BADAF02-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v4 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1212 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 803 insertions(+), 1052 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..34896b303ac 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all objects the same, but to register a +new kind of an object with it, you need to fill in a few callback +functions, see ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..99240679e09 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ - + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, true); + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +841,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +892,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------7349B9D66ED595EF3B1030B4-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v5 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1221 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 811 insertions(+), 1057 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..c67ca603526 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resource the same, and extensions +can define a new kind of resource by defining a new ResourceOwnerFuncs +instance with a few callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..75fda4fcc13 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. + * Remember that an object is owner by a ReourceOwner * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; - - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +746,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +840,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +891,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------1104342304EC6341F5D7C60C Content-Type: text/x-patch; charset=UTF-8; name="0001-resownerbench.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-resownerbench.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v4 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1212 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 803 insertions(+), 1052 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..34896b303ac 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all objects the same, but to register a +new kind of an object with it, you need to fill in a few callback +functions, see ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..99240679e09 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ - + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, true); + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +841,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +892,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------7349B9D66ED595EF3B1030B4-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v5 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1221 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 811 insertions(+), 1057 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..c67ca603526 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resource the same, and extensions +can define a new kind of resource by defining a new ResourceOwnerFuncs +instance with a few callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..75fda4fcc13 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. + * Remember that an object is owner by a ReourceOwner * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; - - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +746,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +840,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +891,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------1104342304EC6341F5D7C60C Content-Type: text/x-patch; charset=UTF-8; name="0001-resownerbench.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-resownerbench.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v3 1/1] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 43 +- src/backend/storage/buffer/localbuf.c | 2 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 98 ++- src/backend/utils/cache/plancache.c | 50 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/cache/syscache.c | 2 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1139 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 15 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 21 files changed, 795 insertions(+), 1020 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c46b8abad12..8b77fb68941 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -739,7 +751,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); isExtend = (blockNum == P_NEW); @@ -1858,7 +1870,7 @@ BufferSync(int flags) WritebackContext wb_context; /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Unless this is a shutdown checkpoint or we have been explicitly told, @@ -2335,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context) */ /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); num_to_scan = bufs_to_lap; num_written = 0; @@ -3488,7 +3500,7 @@ FlushRelationBuffers(Relation rel) } /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3561,7 +3573,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3640,7 +3652,7 @@ FlushDatabaseBuffers(Oid dbid) BufferDesc *bufHdr; /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3738,7 +3750,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4802,3 +4814,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 04b3558ea33..811e0e3a45e 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index 931ed679307..5d2394e6780 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -96,7 +96,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -339,6 +339,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1429,7 +1447,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1611,7 +1629,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1741,7 +1759,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1779,7 +1797,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3635,3 +3653,19 @@ data_sync_elevel(int elevel) { return data_sync_retry ? elevel : PANIC; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index fa2b49c676e..24751c0d3f4 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,66 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* support for catcache refcount management */ +static inline void +ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs); +} +static inline void +ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs); +} + +static inline void +ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs); +} + +static inline void +ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs); +} + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1331,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1432,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1644,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1607,7 +1668,7 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ctlist = NIL; @@ -2062,14 +2123,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) { + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) +{ + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2083,9 +2149,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) +{ + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) { + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..484f14f5524 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +static inline void +ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} +static inline void +ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1254,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1417,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1476,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2230,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index e4dc4ee34ee..d6fe6a5bd5a 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -994,7 +994,7 @@ static const struct cachedesc cacheinfo[] = { } }; -static CatCache *SysCache[SysCacheSize]; + CatCache *SysCache[SysCacheSize]; static bool CacheInitialized = false; diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..890db7d1e66 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context references. +ResourceOwner treates all objects the same, but to register a new kind of +an object with it, you need to fill in a few callback functions, see +ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callabacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..c49758cb20d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) - -/* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. + * ResourceElem represents a reference associated with a resource owner. * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. - * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,45 +153,34 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ - -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* @@ -207,205 +189,227 @@ ResourceArrayInit(ResourceArray *resarr, Datum invalidval) * This is separate from actually inserting a resource because if we run out * of memory, it's critical to do so *before* acquiring the resource. */ -static void -ResourceArrayEnlarge(ResourceArray *resarr) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) + if (owner->narr < RESOWNER_ARRAY_SIZE) return; /* no work needed */ - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; - - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; - - if (olditemsarr != NULL) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. - */ - for (i = 0; i < oldcap; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) + { + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); + } + + /* And release old hash table. */ + pfree(oldhash); } + } - /* And release old array. */ - pfree(olditemsarr); + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - Assert(resarr->nitems < resarr->maxitems); + Assert(owner->nhash < owner->grow_at); } /* - * Add a resource to ResourceArray + * Remember that an object is owner by a ReourceOwner * - * Caller must have previously done ResourceArrayEnlarge() + * Caller must have previously done ResourceOwnerEnlarge() */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 idx; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Append to linear array. */ - idx = resarr->nitems; - } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Remove a resource from ResourceArray + * Forget that an object is owned by a ResourceOwner * - * Returns true on success, false if resource was not found. + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. * - * Note: if same resource ID appears more than once, one instance is removed. + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 i, - idx, - lastidx = resarr->lastidx; + idx; - Assert(value != resarr->invalidval); +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) - { - if (resarr->itemsarr[i] == value) - { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + +#ifdef RESOWNER_STATS + narray_lookups++; +#endif + + return; } } - else + + /* Search hash */ + if (owner->nhash > 0) { - uint32 mask = resarr->capacity - 1; + uint32 mask = owner->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) + for (i = 0; i < owner->capacity; i++) { - if (resarr->itemsarr[idx] == value) + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; } idx = (idx + 1) & mask; } } - return false; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Call the ReleaseResource callback on entries with given 'phase'. */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +static void +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - if (resarr->nitems == 0) - return false; + bool found; + int capacity; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else + /* First handle all the entries in the array. */ + do { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; - - for (;;) + found = false; + for (int i = 0; i < owner->narr; i++) { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } } - } - *value = resarr->itemsarr[resarr->lastidx]; - return true; -} + /* + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. + */ + } while (found && owner->narr > 0); -/* - * Trash a ResourceArray (we don't care about its state after this) - */ -static void -ResourceArrayFree(ResourceArray *resarr) -{ - if (resarr->itemsarr) - pfree(resarr->itemsarr); + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) + { + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } + } + + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } @@ -437,17 +441,10 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name) parent->firstchild = owner; } - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif return owner; } @@ -486,6 +483,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +503,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +518,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +533,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +577,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +600,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - ReleaseCachedPlan(res, true); + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) + { + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + CurrentResourceOwner = save; } @@ -730,19 +652,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +676,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +830,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +881,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..1ed1580c644 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..281f85586fa 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,21 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +static inline void +ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} +static inline void +ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------3CC8131494C3F8E34BADAF02-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v4 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1212 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 803 insertions(+), 1052 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..34896b303ac 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all objects the same, but to register a +new kind of an object with it, you need to fill in a few callback +functions, see ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..99240679e09 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ - + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, true); + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +841,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +892,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------7349B9D66ED595EF3B1030B4-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v5 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1221 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 811 insertions(+), 1057 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..c67ca603526 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resource the same, and extensions +can define a new kind of resource by defining a new ResourceOwnerFuncs +instance with a few callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..75fda4fcc13 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. + * Remember that an object is owner by a ReourceOwner * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; - - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +746,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +840,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +891,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------1104342304EC6341F5D7C60C Content-Type: text/x-patch; charset=UTF-8; name="0001-resownerbench.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-resownerbench.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v5 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1221 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 811 insertions(+), 1057 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..c67ca603526 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resource the same, and extensions +can define a new kind of resource by defining a new ResourceOwnerFuncs +instance with a few callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..75fda4fcc13 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. + * Remember that an object is owner by a ReourceOwner * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; - - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +746,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +840,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +891,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------1104342304EC6341F5D7C60C Content-Type: text/x-patch; charset=UTF-8; name="0001-resownerbench.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-resownerbench.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v3 1/1] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 43 +- src/backend/storage/buffer/localbuf.c | 2 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 98 ++- src/backend/utils/cache/plancache.c | 50 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/cache/syscache.c | 2 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1139 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 15 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 21 files changed, 795 insertions(+), 1020 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c46b8abad12..8b77fb68941 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -739,7 +751,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); isExtend = (blockNum == P_NEW); @@ -1858,7 +1870,7 @@ BufferSync(int flags) WritebackContext wb_context; /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Unless this is a shutdown checkpoint or we have been explicitly told, @@ -2335,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context) */ /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); num_to_scan = bufs_to_lap; num_written = 0; @@ -3488,7 +3500,7 @@ FlushRelationBuffers(Relation rel) } /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3561,7 +3573,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3640,7 +3652,7 @@ FlushDatabaseBuffers(Oid dbid) BufferDesc *bufHdr; /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3738,7 +3750,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4802,3 +4814,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 04b3558ea33..811e0e3a45e 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index 931ed679307..5d2394e6780 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -96,7 +96,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -339,6 +339,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1429,7 +1447,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1611,7 +1629,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1741,7 +1759,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1779,7 +1797,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3635,3 +3653,19 @@ data_sync_elevel(int elevel) { return data_sync_retry ? elevel : PANIC; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index fa2b49c676e..24751c0d3f4 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,66 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* support for catcache refcount management */ +static inline void +ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs); +} +static inline void +ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs); +} + +static inline void +ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs); +} + +static inline void +ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs); +} + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1331,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1432,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1644,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1607,7 +1668,7 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ctlist = NIL; @@ -2062,14 +2123,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) { + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) +{ + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2083,9 +2149,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) +{ + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) { + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..484f14f5524 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +static inline void +ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} +static inline void +ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1254,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1417,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1476,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2230,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index e4dc4ee34ee..d6fe6a5bd5a 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -994,7 +994,7 @@ static const struct cachedesc cacheinfo[] = { } }; -static CatCache *SysCache[SysCacheSize]; + CatCache *SysCache[SysCacheSize]; static bool CacheInitialized = false; diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..890db7d1e66 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context references. +ResourceOwner treates all objects the same, but to register a new kind of +an object with it, you need to fill in a few callback functions, see +ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callabacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..c49758cb20d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) - -/* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. + * ResourceElem represents a reference associated with a resource owner. * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. - * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,45 +153,34 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ - -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* @@ -207,205 +189,227 @@ ResourceArrayInit(ResourceArray *resarr, Datum invalidval) * This is separate from actually inserting a resource because if we run out * of memory, it's critical to do so *before* acquiring the resource. */ -static void -ResourceArrayEnlarge(ResourceArray *resarr) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) + if (owner->narr < RESOWNER_ARRAY_SIZE) return; /* no work needed */ - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; - - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; - - if (olditemsarr != NULL) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. - */ - for (i = 0; i < oldcap; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) + { + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); + } + + /* And release old hash table. */ + pfree(oldhash); } + } - /* And release old array. */ - pfree(olditemsarr); + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - Assert(resarr->nitems < resarr->maxitems); + Assert(owner->nhash < owner->grow_at); } /* - * Add a resource to ResourceArray + * Remember that an object is owner by a ReourceOwner * - * Caller must have previously done ResourceArrayEnlarge() + * Caller must have previously done ResourceOwnerEnlarge() */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 idx; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Append to linear array. */ - idx = resarr->nitems; - } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Remove a resource from ResourceArray + * Forget that an object is owned by a ResourceOwner * - * Returns true on success, false if resource was not found. + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. * - * Note: if same resource ID appears more than once, one instance is removed. + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 i, - idx, - lastidx = resarr->lastidx; + idx; - Assert(value != resarr->invalidval); +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) - { - if (resarr->itemsarr[i] == value) - { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + +#ifdef RESOWNER_STATS + narray_lookups++; +#endif + + return; } } - else + + /* Search hash */ + if (owner->nhash > 0) { - uint32 mask = resarr->capacity - 1; + uint32 mask = owner->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) + for (i = 0; i < owner->capacity; i++) { - if (resarr->itemsarr[idx] == value) + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; } idx = (idx + 1) & mask; } } - return false; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Call the ReleaseResource callback on entries with given 'phase'. */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +static void +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - if (resarr->nitems == 0) - return false; + bool found; + int capacity; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else + /* First handle all the entries in the array. */ + do { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; - - for (;;) + found = false; + for (int i = 0; i < owner->narr; i++) { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } } - } - *value = resarr->itemsarr[resarr->lastidx]; - return true; -} + /* + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. + */ + } while (found && owner->narr > 0); -/* - * Trash a ResourceArray (we don't care about its state after this) - */ -static void -ResourceArrayFree(ResourceArray *resarr) -{ - if (resarr->itemsarr) - pfree(resarr->itemsarr); + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) + { + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } + } + + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } @@ -437,17 +441,10 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name) parent->firstchild = owner; } - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif return owner; } @@ -486,6 +483,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +503,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +518,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +533,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +577,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +600,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - ReleaseCachedPlan(res, true); + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) + { + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + CurrentResourceOwner = save; } @@ -730,19 +652,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +676,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +830,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +881,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..1ed1580c644 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..281f85586fa 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,21 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +static inline void +ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} +static inline void +ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------3CC8131494C3F8E34BADAF02-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v3 1/1] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 43 +- src/backend/storage/buffer/localbuf.c | 2 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 98 ++- src/backend/utils/cache/plancache.c | 50 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/cache/syscache.c | 2 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1139 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 15 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 21 files changed, 795 insertions(+), 1020 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c46b8abad12..8b77fb68941 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -739,7 +751,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); isExtend = (blockNum == P_NEW); @@ -1858,7 +1870,7 @@ BufferSync(int flags) WritebackContext wb_context; /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Unless this is a shutdown checkpoint or we have been explicitly told, @@ -2335,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context) */ /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); num_to_scan = bufs_to_lap; num_written = 0; @@ -3488,7 +3500,7 @@ FlushRelationBuffers(Relation rel) } /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3561,7 +3573,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3640,7 +3652,7 @@ FlushDatabaseBuffers(Oid dbid) BufferDesc *bufHdr; /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3738,7 +3750,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4802,3 +4814,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 04b3558ea33..811e0e3a45e 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index 931ed679307..5d2394e6780 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -96,7 +96,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -339,6 +339,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1429,7 +1447,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1611,7 +1629,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1741,7 +1759,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1779,7 +1797,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3635,3 +3653,19 @@ data_sync_elevel(int elevel) { return data_sync_retry ? elevel : PANIC; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index fa2b49c676e..24751c0d3f4 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,66 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* support for catcache refcount management */ +static inline void +ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs); +} +static inline void +ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs); +} + +static inline void +ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs); +} + +static inline void +ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs); +} + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1331,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1432,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1644,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1607,7 +1668,7 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ctlist = NIL; @@ -2062,14 +2123,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) { + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) +{ + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2083,9 +2149,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) +{ + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) { + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..484f14f5524 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +static inline void +ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} +static inline void +ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1254,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1417,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1476,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2230,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index e4dc4ee34ee..d6fe6a5bd5a 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -994,7 +994,7 @@ static const struct cachedesc cacheinfo[] = { } }; -static CatCache *SysCache[SysCacheSize]; + CatCache *SysCache[SysCacheSize]; static bool CacheInitialized = false; diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..890db7d1e66 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context references. +ResourceOwner treates all objects the same, but to register a new kind of +an object with it, you need to fill in a few callback functions, see +ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callabacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..c49758cb20d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) - -/* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. + * ResourceElem represents a reference associated with a resource owner. * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. - * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,45 +153,34 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ - -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* @@ -207,205 +189,227 @@ ResourceArrayInit(ResourceArray *resarr, Datum invalidval) * This is separate from actually inserting a resource because if we run out * of memory, it's critical to do so *before* acquiring the resource. */ -static void -ResourceArrayEnlarge(ResourceArray *resarr) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) + if (owner->narr < RESOWNER_ARRAY_SIZE) return; /* no work needed */ - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; - - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; - - if (olditemsarr != NULL) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. - */ - for (i = 0; i < oldcap; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) + { + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); + } + + /* And release old hash table. */ + pfree(oldhash); } + } - /* And release old array. */ - pfree(olditemsarr); + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - Assert(resarr->nitems < resarr->maxitems); + Assert(owner->nhash < owner->grow_at); } /* - * Add a resource to ResourceArray + * Remember that an object is owner by a ReourceOwner * - * Caller must have previously done ResourceArrayEnlarge() + * Caller must have previously done ResourceOwnerEnlarge() */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 idx; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Append to linear array. */ - idx = resarr->nitems; - } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Remove a resource from ResourceArray + * Forget that an object is owned by a ResourceOwner * - * Returns true on success, false if resource was not found. + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. * - * Note: if same resource ID appears more than once, one instance is removed. + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 i, - idx, - lastidx = resarr->lastidx; + idx; - Assert(value != resarr->invalidval); +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) - { - if (resarr->itemsarr[i] == value) - { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + +#ifdef RESOWNER_STATS + narray_lookups++; +#endif + + return; } } - else + + /* Search hash */ + if (owner->nhash > 0) { - uint32 mask = resarr->capacity - 1; + uint32 mask = owner->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) + for (i = 0; i < owner->capacity; i++) { - if (resarr->itemsarr[idx] == value) + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; } idx = (idx + 1) & mask; } } - return false; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Call the ReleaseResource callback on entries with given 'phase'. */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +static void +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - if (resarr->nitems == 0) - return false; + bool found; + int capacity; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else + /* First handle all the entries in the array. */ + do { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; - - for (;;) + found = false; + for (int i = 0; i < owner->narr; i++) { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } } - } - *value = resarr->itemsarr[resarr->lastidx]; - return true; -} + /* + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. + */ + } while (found && owner->narr > 0); -/* - * Trash a ResourceArray (we don't care about its state after this) - */ -static void -ResourceArrayFree(ResourceArray *resarr) -{ - if (resarr->itemsarr) - pfree(resarr->itemsarr); + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) + { + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } + } + + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } @@ -437,17 +441,10 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name) parent->firstchild = owner; } - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif return owner; } @@ -486,6 +483,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +503,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +518,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +533,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +577,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +600,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - ReleaseCachedPlan(res, true); + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) + { + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + CurrentResourceOwner = save; } @@ -730,19 +652,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +676,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +830,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +881,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..1ed1580c644 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..281f85586fa 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,21 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +static inline void +ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} +static inline void +ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------3CC8131494C3F8E34BADAF02-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v4 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1212 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 803 insertions(+), 1052 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..34896b303ac 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all objects the same, but to register a +new kind of an object with it, you need to fill in a few callback +functions, see ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..99240679e09 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ - + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, true); + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +841,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +892,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------7349B9D66ED595EF3B1030B4-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v5 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1221 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 811 insertions(+), 1057 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..c67ca603526 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resource the same, and extensions +can define a new kind of resource by defining a new ResourceOwnerFuncs +instance with a few callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..75fda4fcc13 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. + * Remember that an object is owner by a ReourceOwner * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; - - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +746,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +840,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +891,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------1104342304EC6341F5D7C60C Content-Type: text/x-patch; charset=UTF-8; name="0001-resownerbench.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-resownerbench.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v3 1/1] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 43 +- src/backend/storage/buffer/localbuf.c | 2 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 98 ++- src/backend/utils/cache/plancache.c | 50 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/cache/syscache.c | 2 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1139 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 15 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 21 files changed, 795 insertions(+), 1020 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c46b8abad12..8b77fb68941 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -739,7 +751,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); isExtend = (blockNum == P_NEW); @@ -1858,7 +1870,7 @@ BufferSync(int flags) WritebackContext wb_context; /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Unless this is a shutdown checkpoint or we have been explicitly told, @@ -2335,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context) */ /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); num_to_scan = bufs_to_lap; num_written = 0; @@ -3488,7 +3500,7 @@ FlushRelationBuffers(Relation rel) } /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3561,7 +3573,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3640,7 +3652,7 @@ FlushDatabaseBuffers(Oid dbid) BufferDesc *bufHdr; /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3738,7 +3750,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4802,3 +4814,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 04b3558ea33..811e0e3a45e 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index 931ed679307..5d2394e6780 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -96,7 +96,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -339,6 +339,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1429,7 +1447,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1611,7 +1629,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1741,7 +1759,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1779,7 +1797,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3635,3 +3653,19 @@ data_sync_elevel(int elevel) { return data_sync_retry ? elevel : PANIC; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index fa2b49c676e..24751c0d3f4 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,66 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* support for catcache refcount management */ +static inline void +ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs); +} +static inline void +ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs); +} + +static inline void +ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs); +} + +static inline void +ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs); +} + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1331,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1432,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1644,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1607,7 +1668,7 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ctlist = NIL; @@ -2062,14 +2123,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) { + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) +{ + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2083,9 +2149,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) +{ + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) { + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..484f14f5524 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +static inline void +ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} +static inline void +ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1254,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1417,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1476,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2230,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index e4dc4ee34ee..d6fe6a5bd5a 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -994,7 +994,7 @@ static const struct cachedesc cacheinfo[] = { } }; -static CatCache *SysCache[SysCacheSize]; + CatCache *SysCache[SysCacheSize]; static bool CacheInitialized = false; diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..890db7d1e66 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context references. +ResourceOwner treates all objects the same, but to register a new kind of +an object with it, you need to fill in a few callback functions, see +ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callabacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..c49758cb20d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) - -/* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. + * ResourceElem represents a reference associated with a resource owner. * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. - * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,45 +153,34 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ - -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* @@ -207,205 +189,227 @@ ResourceArrayInit(ResourceArray *resarr, Datum invalidval) * This is separate from actually inserting a resource because if we run out * of memory, it's critical to do so *before* acquiring the resource. */ -static void -ResourceArrayEnlarge(ResourceArray *resarr) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) + if (owner->narr < RESOWNER_ARRAY_SIZE) return; /* no work needed */ - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; - - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; - - if (olditemsarr != NULL) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. - */ - for (i = 0; i < oldcap; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) + { + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); + } + + /* And release old hash table. */ + pfree(oldhash); } + } - /* And release old array. */ - pfree(olditemsarr); + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - Assert(resarr->nitems < resarr->maxitems); + Assert(owner->nhash < owner->grow_at); } /* - * Add a resource to ResourceArray + * Remember that an object is owner by a ReourceOwner * - * Caller must have previously done ResourceArrayEnlarge() + * Caller must have previously done ResourceOwnerEnlarge() */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 idx; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Append to linear array. */ - idx = resarr->nitems; - } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Remove a resource from ResourceArray + * Forget that an object is owned by a ResourceOwner * - * Returns true on success, false if resource was not found. + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. * - * Note: if same resource ID appears more than once, one instance is removed. + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 i, - idx, - lastidx = resarr->lastidx; + idx; - Assert(value != resarr->invalidval); +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) - { - if (resarr->itemsarr[i] == value) - { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + +#ifdef RESOWNER_STATS + narray_lookups++; +#endif + + return; } } - else + + /* Search hash */ + if (owner->nhash > 0) { - uint32 mask = resarr->capacity - 1; + uint32 mask = owner->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) + for (i = 0; i < owner->capacity; i++) { - if (resarr->itemsarr[idx] == value) + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; } idx = (idx + 1) & mask; } } - return false; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Call the ReleaseResource callback on entries with given 'phase'. */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +static void +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - if (resarr->nitems == 0) - return false; + bool found; + int capacity; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else + /* First handle all the entries in the array. */ + do { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; - - for (;;) + found = false; + for (int i = 0; i < owner->narr; i++) { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } } - } - *value = resarr->itemsarr[resarr->lastidx]; - return true; -} + /* + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. + */ + } while (found && owner->narr > 0); -/* - * Trash a ResourceArray (we don't care about its state after this) - */ -static void -ResourceArrayFree(ResourceArray *resarr) -{ - if (resarr->itemsarr) - pfree(resarr->itemsarr); + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) + { + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } + } + + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } @@ -437,17 +441,10 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name) parent->firstchild = owner; } - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif return owner; } @@ -486,6 +483,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +503,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +518,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +533,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +577,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +600,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - ReleaseCachedPlan(res, true); + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) + { + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + CurrentResourceOwner = save; } @@ -730,19 +652,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +676,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +830,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +881,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..1ed1580c644 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..281f85586fa 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,21 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +static inline void +ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} +static inline void +ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------3CC8131494C3F8E34BADAF02-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v4 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1212 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 803 insertions(+), 1052 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..34896b303ac 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all objects the same, but to register a +new kind of an object with it, you need to fill in a few callback +functions, see ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..99240679e09 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ - + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, true); + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +841,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +892,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------7349B9D66ED595EF3B1030B4-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v3 1/1] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 43 +- src/backend/storage/buffer/localbuf.c | 2 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 98 ++- src/backend/utils/cache/plancache.c | 50 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/cache/syscache.c | 2 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1139 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 15 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 21 files changed, 795 insertions(+), 1020 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c46b8abad12..8b77fb68941 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -739,7 +751,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); isExtend = (blockNum == P_NEW); @@ -1858,7 +1870,7 @@ BufferSync(int flags) WritebackContext wb_context; /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Unless this is a shutdown checkpoint or we have been explicitly told, @@ -2335,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context) */ /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); num_to_scan = bufs_to_lap; num_written = 0; @@ -3488,7 +3500,7 @@ FlushRelationBuffers(Relation rel) } /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3561,7 +3573,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3640,7 +3652,7 @@ FlushDatabaseBuffers(Oid dbid) BufferDesc *bufHdr; /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3738,7 +3750,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4802,3 +4814,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 04b3558ea33..811e0e3a45e 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index 931ed679307..5d2394e6780 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -96,7 +96,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -339,6 +339,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1429,7 +1447,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1611,7 +1629,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1741,7 +1759,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1779,7 +1797,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3635,3 +3653,19 @@ data_sync_elevel(int elevel) { return data_sync_retry ? elevel : PANIC; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index fa2b49c676e..24751c0d3f4 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,66 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* support for catcache refcount management */ +static inline void +ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs); +} +static inline void +ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs); +} + +static inline void +ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs); +} + +static inline void +ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs); +} + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1331,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1432,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1644,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1607,7 +1668,7 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ctlist = NIL; @@ -2062,14 +2123,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) { + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) +{ + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2083,9 +2149,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) +{ + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) { + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..484f14f5524 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +static inline void +ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} +static inline void +ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1254,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1417,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1476,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2230,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index e4dc4ee34ee..d6fe6a5bd5a 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -994,7 +994,7 @@ static const struct cachedesc cacheinfo[] = { } }; -static CatCache *SysCache[SysCacheSize]; + CatCache *SysCache[SysCacheSize]; static bool CacheInitialized = false; diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..890db7d1e66 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context references. +ResourceOwner treates all objects the same, but to register a new kind of +an object with it, you need to fill in a few callback functions, see +ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callabacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..c49758cb20d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) - -/* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. + * ResourceElem represents a reference associated with a resource owner. * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. - * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,45 +153,34 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ - -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* @@ -207,205 +189,227 @@ ResourceArrayInit(ResourceArray *resarr, Datum invalidval) * This is separate from actually inserting a resource because if we run out * of memory, it's critical to do so *before* acquiring the resource. */ -static void -ResourceArrayEnlarge(ResourceArray *resarr) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) + if (owner->narr < RESOWNER_ARRAY_SIZE) return; /* no work needed */ - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; - - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; - - if (olditemsarr != NULL) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. - */ - for (i = 0; i < oldcap; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) + { + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); + } + + /* And release old hash table. */ + pfree(oldhash); } + } - /* And release old array. */ - pfree(olditemsarr); + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - Assert(resarr->nitems < resarr->maxitems); + Assert(owner->nhash < owner->grow_at); } /* - * Add a resource to ResourceArray + * Remember that an object is owner by a ReourceOwner * - * Caller must have previously done ResourceArrayEnlarge() + * Caller must have previously done ResourceOwnerEnlarge() */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 idx; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Append to linear array. */ - idx = resarr->nitems; - } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Remove a resource from ResourceArray + * Forget that an object is owned by a ResourceOwner * - * Returns true on success, false if resource was not found. + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. * - * Note: if same resource ID appears more than once, one instance is removed. + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 i, - idx, - lastidx = resarr->lastidx; + idx; - Assert(value != resarr->invalidval); +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) - { - if (resarr->itemsarr[i] == value) - { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + +#ifdef RESOWNER_STATS + narray_lookups++; +#endif + + return; } } - else + + /* Search hash */ + if (owner->nhash > 0) { - uint32 mask = resarr->capacity - 1; + uint32 mask = owner->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) + for (i = 0; i < owner->capacity; i++) { - if (resarr->itemsarr[idx] == value) + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; } idx = (idx + 1) & mask; } } - return false; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Call the ReleaseResource callback on entries with given 'phase'. */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +static void +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - if (resarr->nitems == 0) - return false; + bool found; + int capacity; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else + /* First handle all the entries in the array. */ + do { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; - - for (;;) + found = false; + for (int i = 0; i < owner->narr; i++) { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } } - } - *value = resarr->itemsarr[resarr->lastidx]; - return true; -} + /* + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. + */ + } while (found && owner->narr > 0); -/* - * Trash a ResourceArray (we don't care about its state after this) - */ -static void -ResourceArrayFree(ResourceArray *resarr) -{ - if (resarr->itemsarr) - pfree(resarr->itemsarr); + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) + { + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } + } + + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } @@ -437,17 +441,10 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name) parent->firstchild = owner; } - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif return owner; } @@ -486,6 +483,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +503,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +518,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +533,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +577,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +600,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - ReleaseCachedPlan(res, true); + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) + { + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + CurrentResourceOwner = save; } @@ -730,19 +652,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +676,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +830,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +881,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..1ed1580c644 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..281f85586fa 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,21 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +static inline void +ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} +static inline void +ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------3CC8131494C3F8E34BADAF02-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v4 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1212 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 803 insertions(+), 1052 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..34896b303ac 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all objects the same, but to register a +new kind of an object with it, you need to fill in a few callback +functions, see ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..99240679e09 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ - + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, true); + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +841,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +892,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------7349B9D66ED595EF3B1030B4-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v5 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1221 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 811 insertions(+), 1057 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..c67ca603526 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resource the same, and extensions +can define a new kind of resource by defining a new ResourceOwnerFuncs +instance with a few callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..75fda4fcc13 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. + * Remember that an object is owner by a ReourceOwner * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; - - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +746,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +840,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +891,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------1104342304EC6341F5D7C60C Content-Type: text/x-patch; charset=UTF-8; name="0001-resownerbench.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-resownerbench.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v5 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1221 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 811 insertions(+), 1057 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..c67ca603526 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resource the same, and extensions +can define a new kind of resource by defining a new ResourceOwnerFuncs +instance with a few callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..75fda4fcc13 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. + * Remember that an object is owner by a ReourceOwner * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; - - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +746,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +840,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +891,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------1104342304EC6341F5D7C60C Content-Type: text/x-patch; charset=UTF-8; name="0001-resownerbench.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-resownerbench.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v4 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1212 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 803 insertions(+), 1052 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..34896b303ac 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all objects the same, but to register a +new kind of an object with it, you need to fill in a few callback +functions, see ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..99240679e09 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ - + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, true); + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +841,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +892,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------7349B9D66ED595EF3B1030B4-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v3 1/1] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 43 +- src/backend/storage/buffer/localbuf.c | 2 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 98 ++- src/backend/utils/cache/plancache.c | 50 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/cache/syscache.c | 2 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1139 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 15 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 21 files changed, 795 insertions(+), 1020 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c46b8abad12..8b77fb68941 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -739,7 +751,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); isExtend = (blockNum == P_NEW); @@ -1858,7 +1870,7 @@ BufferSync(int flags) WritebackContext wb_context; /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Unless this is a shutdown checkpoint or we have been explicitly told, @@ -2335,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context) */ /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); num_to_scan = bufs_to_lap; num_written = 0; @@ -3488,7 +3500,7 @@ FlushRelationBuffers(Relation rel) } /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3561,7 +3573,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3640,7 +3652,7 @@ FlushDatabaseBuffers(Oid dbid) BufferDesc *bufHdr; /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3738,7 +3750,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4802,3 +4814,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 04b3558ea33..811e0e3a45e 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index 931ed679307..5d2394e6780 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -96,7 +96,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -339,6 +339,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1429,7 +1447,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1611,7 +1629,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1741,7 +1759,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1779,7 +1797,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3635,3 +3653,19 @@ data_sync_elevel(int elevel) { return data_sync_retry ? elevel : PANIC; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index fa2b49c676e..24751c0d3f4 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,66 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* support for catcache refcount management */ +static inline void +ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs); +} +static inline void +ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs); +} + +static inline void +ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs); +} + +static inline void +ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs); +} + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1331,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1432,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1644,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1607,7 +1668,7 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ctlist = NIL; @@ -2062,14 +2123,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) { + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) +{ + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2083,9 +2149,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) +{ + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) { + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..484f14f5524 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +static inline void +ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} +static inline void +ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1254,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1417,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1476,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2230,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index e4dc4ee34ee..d6fe6a5bd5a 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -994,7 +994,7 @@ static const struct cachedesc cacheinfo[] = { } }; -static CatCache *SysCache[SysCacheSize]; + CatCache *SysCache[SysCacheSize]; static bool CacheInitialized = false; diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..890db7d1e66 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context references. +ResourceOwner treates all objects the same, but to register a new kind of +an object with it, you need to fill in a few callback functions, see +ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callabacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..c49758cb20d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) - -/* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. + * ResourceElem represents a reference associated with a resource owner. * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. - * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,45 +153,34 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ - -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* @@ -207,205 +189,227 @@ ResourceArrayInit(ResourceArray *resarr, Datum invalidval) * This is separate from actually inserting a resource because if we run out * of memory, it's critical to do so *before* acquiring the resource. */ -static void -ResourceArrayEnlarge(ResourceArray *resarr) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) + if (owner->narr < RESOWNER_ARRAY_SIZE) return; /* no work needed */ - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; - - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; - - if (olditemsarr != NULL) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. - */ - for (i = 0; i < oldcap; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) + { + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); + } + + /* And release old hash table. */ + pfree(oldhash); } + } - /* And release old array. */ - pfree(olditemsarr); + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - Assert(resarr->nitems < resarr->maxitems); + Assert(owner->nhash < owner->grow_at); } /* - * Add a resource to ResourceArray + * Remember that an object is owner by a ReourceOwner * - * Caller must have previously done ResourceArrayEnlarge() + * Caller must have previously done ResourceOwnerEnlarge() */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 idx; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Append to linear array. */ - idx = resarr->nitems; - } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Remove a resource from ResourceArray + * Forget that an object is owned by a ResourceOwner * - * Returns true on success, false if resource was not found. + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. * - * Note: if same resource ID appears more than once, one instance is removed. + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 i, - idx, - lastidx = resarr->lastidx; + idx; - Assert(value != resarr->invalidval); +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) - { - if (resarr->itemsarr[i] == value) - { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + +#ifdef RESOWNER_STATS + narray_lookups++; +#endif + + return; } } - else + + /* Search hash */ + if (owner->nhash > 0) { - uint32 mask = resarr->capacity - 1; + uint32 mask = owner->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) + for (i = 0; i < owner->capacity; i++) { - if (resarr->itemsarr[idx] == value) + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; } idx = (idx + 1) & mask; } } - return false; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Call the ReleaseResource callback on entries with given 'phase'. */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +static void +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - if (resarr->nitems == 0) - return false; + bool found; + int capacity; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else + /* First handle all the entries in the array. */ + do { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; - - for (;;) + found = false; + for (int i = 0; i < owner->narr; i++) { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } } - } - *value = resarr->itemsarr[resarr->lastidx]; - return true; -} + /* + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. + */ + } while (found && owner->narr > 0); -/* - * Trash a ResourceArray (we don't care about its state after this) - */ -static void -ResourceArrayFree(ResourceArray *resarr) -{ - if (resarr->itemsarr) - pfree(resarr->itemsarr); + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) + { + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } + } + + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } @@ -437,17 +441,10 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name) parent->firstchild = owner; } - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif return owner; } @@ -486,6 +483,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +503,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +518,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +533,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +577,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +600,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - ReleaseCachedPlan(res, true); + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) + { + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + CurrentResourceOwner = save; } @@ -730,19 +652,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +676,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +830,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +881,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..1ed1580c644 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..281f85586fa 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,21 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +static inline void +ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} +static inline void +ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------3CC8131494C3F8E34BADAF02-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v4 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1212 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 803 insertions(+), 1052 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..34896b303ac 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all objects the same, but to register a +new kind of an object with it, you need to fill in a few callback +functions, see ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..99240679e09 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ - + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, true); + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +841,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +892,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------7349B9D66ED595EF3B1030B4-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v5 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1221 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 811 insertions(+), 1057 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..c67ca603526 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resource the same, and extensions +can define a new kind of resource by defining a new ResourceOwnerFuncs +instance with a few callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..75fda4fcc13 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. + * Remember that an object is owner by a ReourceOwner * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; - - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +746,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +840,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +891,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------1104342304EC6341F5D7C60C Content-Type: text/x-patch; charset=UTF-8; name="0001-resownerbench.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-resownerbench.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v5 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1221 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 811 insertions(+), 1057 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..c67ca603526 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resource the same, and extensions +can define a new kind of resource by defining a new ResourceOwnerFuncs +instance with a few callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..75fda4fcc13 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. + * Remember that an object is owner by a ReourceOwner * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; - - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +746,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +840,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +891,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------1104342304EC6341F5D7C60C Content-Type: text/x-patch; charset=UTF-8; name="0001-resownerbench.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-resownerbench.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v3 1/1] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 43 +- src/backend/storage/buffer/localbuf.c | 2 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 98 ++- src/backend/utils/cache/plancache.c | 50 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/cache/syscache.c | 2 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1139 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 15 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 21 files changed, 795 insertions(+), 1020 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c46b8abad12..8b77fb68941 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -739,7 +751,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); isExtend = (blockNum == P_NEW); @@ -1858,7 +1870,7 @@ BufferSync(int flags) WritebackContext wb_context; /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Unless this is a shutdown checkpoint or we have been explicitly told, @@ -2335,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context) */ /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); num_to_scan = bufs_to_lap; num_written = 0; @@ -3488,7 +3500,7 @@ FlushRelationBuffers(Relation rel) } /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3561,7 +3573,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3640,7 +3652,7 @@ FlushDatabaseBuffers(Oid dbid) BufferDesc *bufHdr; /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3738,7 +3750,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4802,3 +4814,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 04b3558ea33..811e0e3a45e 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index 931ed679307..5d2394e6780 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -96,7 +96,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -339,6 +339,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1429,7 +1447,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1611,7 +1629,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1741,7 +1759,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1779,7 +1797,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3635,3 +3653,19 @@ data_sync_elevel(int elevel) { return data_sync_retry ? elevel : PANIC; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index fa2b49c676e..24751c0d3f4 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,66 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* support for catcache refcount management */ +static inline void +ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs); +} +static inline void +ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs); +} + +static inline void +ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs); +} + +static inline void +ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs); +} + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1331,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1432,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1644,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1607,7 +1668,7 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ctlist = NIL; @@ -2062,14 +2123,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) { + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) +{ + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2083,9 +2149,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) +{ + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) { + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..484f14f5524 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +static inline void +ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} +static inline void +ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1254,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1417,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1476,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2230,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index e4dc4ee34ee..d6fe6a5bd5a 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -994,7 +994,7 @@ static const struct cachedesc cacheinfo[] = { } }; -static CatCache *SysCache[SysCacheSize]; + CatCache *SysCache[SysCacheSize]; static bool CacheInitialized = false; diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..890db7d1e66 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context references. +ResourceOwner treates all objects the same, but to register a new kind of +an object with it, you need to fill in a few callback functions, see +ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callabacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..c49758cb20d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) - -/* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. + * ResourceElem represents a reference associated with a resource owner. * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. - * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,45 +153,34 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ - -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* @@ -207,205 +189,227 @@ ResourceArrayInit(ResourceArray *resarr, Datum invalidval) * This is separate from actually inserting a resource because if we run out * of memory, it's critical to do so *before* acquiring the resource. */ -static void -ResourceArrayEnlarge(ResourceArray *resarr) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) + if (owner->narr < RESOWNER_ARRAY_SIZE) return; /* no work needed */ - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; - - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; - - if (olditemsarr != NULL) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. - */ - for (i = 0; i < oldcap; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) + { + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); + } + + /* And release old hash table. */ + pfree(oldhash); } + } - /* And release old array. */ - pfree(olditemsarr); + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - Assert(resarr->nitems < resarr->maxitems); + Assert(owner->nhash < owner->grow_at); } /* - * Add a resource to ResourceArray + * Remember that an object is owner by a ReourceOwner * - * Caller must have previously done ResourceArrayEnlarge() + * Caller must have previously done ResourceOwnerEnlarge() */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 idx; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Append to linear array. */ - idx = resarr->nitems; - } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Remove a resource from ResourceArray + * Forget that an object is owned by a ResourceOwner * - * Returns true on success, false if resource was not found. + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. * - * Note: if same resource ID appears more than once, one instance is removed. + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 i, - idx, - lastidx = resarr->lastidx; + idx; - Assert(value != resarr->invalidval); +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) - { - if (resarr->itemsarr[i] == value) - { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + +#ifdef RESOWNER_STATS + narray_lookups++; +#endif + + return; } } - else + + /* Search hash */ + if (owner->nhash > 0) { - uint32 mask = resarr->capacity - 1; + uint32 mask = owner->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) + for (i = 0; i < owner->capacity; i++) { - if (resarr->itemsarr[idx] == value) + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; } idx = (idx + 1) & mask; } } - return false; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Call the ReleaseResource callback on entries with given 'phase'. */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +static void +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - if (resarr->nitems == 0) - return false; + bool found; + int capacity; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else + /* First handle all the entries in the array. */ + do { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; - - for (;;) + found = false; + for (int i = 0; i < owner->narr; i++) { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } } - } - *value = resarr->itemsarr[resarr->lastidx]; - return true; -} + /* + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. + */ + } while (found && owner->narr > 0); -/* - * Trash a ResourceArray (we don't care about its state after this) - */ -static void -ResourceArrayFree(ResourceArray *resarr) -{ - if (resarr->itemsarr) - pfree(resarr->itemsarr); + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) + { + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } + } + + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } @@ -437,17 +441,10 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name) parent->firstchild = owner; } - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif return owner; } @@ -486,6 +483,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +503,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +518,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +533,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +577,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +600,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - ReleaseCachedPlan(res, true); + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) + { + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + CurrentResourceOwner = save; } @@ -730,19 +652,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +676,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +830,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +881,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..1ed1580c644 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..281f85586fa 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,21 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +static inline void +ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} +static inline void +ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------3CC8131494C3F8E34BADAF02-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v3 1/1] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 43 +- src/backend/storage/buffer/localbuf.c | 2 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 98 ++- src/backend/utils/cache/plancache.c | 50 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/cache/syscache.c | 2 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1139 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 15 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 21 files changed, 795 insertions(+), 1020 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c46b8abad12..8b77fb68941 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -739,7 +751,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); isExtend = (blockNum == P_NEW); @@ -1858,7 +1870,7 @@ BufferSync(int flags) WritebackContext wb_context; /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Unless this is a shutdown checkpoint or we have been explicitly told, @@ -2335,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context) */ /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); num_to_scan = bufs_to_lap; num_written = 0; @@ -3488,7 +3500,7 @@ FlushRelationBuffers(Relation rel) } /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3561,7 +3573,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3640,7 +3652,7 @@ FlushDatabaseBuffers(Oid dbid) BufferDesc *bufHdr; /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3738,7 +3750,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4802,3 +4814,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 04b3558ea33..811e0e3a45e 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index 931ed679307..5d2394e6780 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -96,7 +96,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -339,6 +339,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1429,7 +1447,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1611,7 +1629,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1741,7 +1759,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1779,7 +1797,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3635,3 +3653,19 @@ data_sync_elevel(int elevel) { return data_sync_retry ? elevel : PANIC; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index fa2b49c676e..24751c0d3f4 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,66 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* support for catcache refcount management */ +static inline void +ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs); +} +static inline void +ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs); +} + +static inline void +ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs); +} + +static inline void +ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs); +} + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1331,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1432,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1644,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1607,7 +1668,7 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ctlist = NIL; @@ -2062,14 +2123,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) { + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) +{ + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2083,9 +2149,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) +{ + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) { + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..484f14f5524 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +static inline void +ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} +static inline void +ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1254,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1417,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1476,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2230,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index e4dc4ee34ee..d6fe6a5bd5a 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -994,7 +994,7 @@ static const struct cachedesc cacheinfo[] = { } }; -static CatCache *SysCache[SysCacheSize]; + CatCache *SysCache[SysCacheSize]; static bool CacheInitialized = false; diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..890db7d1e66 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context references. +ResourceOwner treates all objects the same, but to register a new kind of +an object with it, you need to fill in a few callback functions, see +ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callabacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..c49758cb20d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) - -/* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. + * ResourceElem represents a reference associated with a resource owner. * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. - * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,45 +153,34 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ - -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* @@ -207,205 +189,227 @@ ResourceArrayInit(ResourceArray *resarr, Datum invalidval) * This is separate from actually inserting a resource because if we run out * of memory, it's critical to do so *before* acquiring the resource. */ -static void -ResourceArrayEnlarge(ResourceArray *resarr) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) + if (owner->narr < RESOWNER_ARRAY_SIZE) return; /* no work needed */ - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; - - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; - - if (olditemsarr != NULL) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. - */ - for (i = 0; i < oldcap; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) + { + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); + } + + /* And release old hash table. */ + pfree(oldhash); } + } - /* And release old array. */ - pfree(olditemsarr); + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - Assert(resarr->nitems < resarr->maxitems); + Assert(owner->nhash < owner->grow_at); } /* - * Add a resource to ResourceArray + * Remember that an object is owner by a ReourceOwner * - * Caller must have previously done ResourceArrayEnlarge() + * Caller must have previously done ResourceOwnerEnlarge() */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 idx; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Append to linear array. */ - idx = resarr->nitems; - } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Remove a resource from ResourceArray + * Forget that an object is owned by a ResourceOwner * - * Returns true on success, false if resource was not found. + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. * - * Note: if same resource ID appears more than once, one instance is removed. + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 i, - idx, - lastidx = resarr->lastidx; + idx; - Assert(value != resarr->invalidval); +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) - { - if (resarr->itemsarr[i] == value) - { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + +#ifdef RESOWNER_STATS + narray_lookups++; +#endif + + return; } } - else + + /* Search hash */ + if (owner->nhash > 0) { - uint32 mask = resarr->capacity - 1; + uint32 mask = owner->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) + for (i = 0; i < owner->capacity; i++) { - if (resarr->itemsarr[idx] == value) + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; } idx = (idx + 1) & mask; } } - return false; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Call the ReleaseResource callback on entries with given 'phase'. */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +static void +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - if (resarr->nitems == 0) - return false; + bool found; + int capacity; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else + /* First handle all the entries in the array. */ + do { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; - - for (;;) + found = false; + for (int i = 0; i < owner->narr; i++) { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } } - } - *value = resarr->itemsarr[resarr->lastidx]; - return true; -} + /* + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. + */ + } while (found && owner->narr > 0); -/* - * Trash a ResourceArray (we don't care about its state after this) - */ -static void -ResourceArrayFree(ResourceArray *resarr) -{ - if (resarr->itemsarr) - pfree(resarr->itemsarr); + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) + { + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } + } + + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } @@ -437,17 +441,10 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name) parent->firstchild = owner; } - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif return owner; } @@ -486,6 +483,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +503,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +518,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +533,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +577,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +600,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - ReleaseCachedPlan(res, true); + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) + { + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + CurrentResourceOwner = save; } @@ -730,19 +652,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +676,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +830,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +881,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..1ed1580c644 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..281f85586fa 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,21 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +static inline void +ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} +static inline void +ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------3CC8131494C3F8E34BADAF02-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v4 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1212 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 803 insertions(+), 1052 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..34896b303ac 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all objects the same, but to register a +new kind of an object with it, you need to fill in a few callback +functions, see ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..99240679e09 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ - + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, true); + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +841,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +892,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------7349B9D66ED595EF3B1030B4-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v5 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1221 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 811 insertions(+), 1057 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..c67ca603526 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resource the same, and extensions +can define a new kind of resource by defining a new ResourceOwnerFuncs +instance with a few callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..75fda4fcc13 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. + * Remember that an object is owner by a ReourceOwner * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; - - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +746,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +840,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +891,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------1104342304EC6341F5D7C60C Content-Type: text/x-patch; charset=UTF-8; name="0001-resownerbench.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-resownerbench.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v4 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1212 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 803 insertions(+), 1052 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..34896b303ac 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all objects the same, but to register a +new kind of an object with it, you need to fill in a few callback +functions, see ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..99240679e09 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ - + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, true); + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +841,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +892,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------7349B9D66ED595EF3B1030B4-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v5 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1221 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 811 insertions(+), 1057 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..c67ca603526 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resource the same, and extensions +can define a new kind of resource by defining a new ResourceOwnerFuncs +instance with a few callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..75fda4fcc13 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. + * Remember that an object is owner by a ReourceOwner * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; - - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +746,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +840,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +891,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------1104342304EC6341F5D7C60C Content-Type: text/x-patch; charset=UTF-8; name="0001-resownerbench.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-resownerbench.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v3 1/1] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 43 +- src/backend/storage/buffer/localbuf.c | 2 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 98 ++- src/backend/utils/cache/plancache.c | 50 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/cache/syscache.c | 2 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1139 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 15 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 21 files changed, 795 insertions(+), 1020 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c46b8abad12..8b77fb68941 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -739,7 +751,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); isExtend = (blockNum == P_NEW); @@ -1858,7 +1870,7 @@ BufferSync(int flags) WritebackContext wb_context; /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Unless this is a shutdown checkpoint or we have been explicitly told, @@ -2335,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context) */ /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); num_to_scan = bufs_to_lap; num_written = 0; @@ -3488,7 +3500,7 @@ FlushRelationBuffers(Relation rel) } /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3561,7 +3573,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3640,7 +3652,7 @@ FlushDatabaseBuffers(Oid dbid) BufferDesc *bufHdr; /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3738,7 +3750,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4802,3 +4814,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 04b3558ea33..811e0e3a45e 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index 931ed679307..5d2394e6780 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -96,7 +96,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -339,6 +339,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1429,7 +1447,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1611,7 +1629,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1741,7 +1759,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1779,7 +1797,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3635,3 +3653,19 @@ data_sync_elevel(int elevel) { return data_sync_retry ? elevel : PANIC; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index fa2b49c676e..24751c0d3f4 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,66 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* support for catcache refcount management */ +static inline void +ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs); +} +static inline void +ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs); +} + +static inline void +ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs); +} + +static inline void +ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs); +} + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1331,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1432,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1644,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1607,7 +1668,7 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ctlist = NIL; @@ -2062,14 +2123,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) { + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) +{ + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2083,9 +2149,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) +{ + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) { + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..484f14f5524 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +static inline void +ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} +static inline void +ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1254,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1417,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1476,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2230,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index e4dc4ee34ee..d6fe6a5bd5a 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -994,7 +994,7 @@ static const struct cachedesc cacheinfo[] = { } }; -static CatCache *SysCache[SysCacheSize]; + CatCache *SysCache[SysCacheSize]; static bool CacheInitialized = false; diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..890db7d1e66 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context references. +ResourceOwner treates all objects the same, but to register a new kind of +an object with it, you need to fill in a few callback functions, see +ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callabacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..c49758cb20d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) - -/* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. + * ResourceElem represents a reference associated with a resource owner. * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. - * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,45 +153,34 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ - -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* @@ -207,205 +189,227 @@ ResourceArrayInit(ResourceArray *resarr, Datum invalidval) * This is separate from actually inserting a resource because if we run out * of memory, it's critical to do so *before* acquiring the resource. */ -static void -ResourceArrayEnlarge(ResourceArray *resarr) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) + if (owner->narr < RESOWNER_ARRAY_SIZE) return; /* no work needed */ - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; - - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; - - if (olditemsarr != NULL) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. - */ - for (i = 0; i < oldcap; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) + { + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); + } + + /* And release old hash table. */ + pfree(oldhash); } + } - /* And release old array. */ - pfree(olditemsarr); + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - Assert(resarr->nitems < resarr->maxitems); + Assert(owner->nhash < owner->grow_at); } /* - * Add a resource to ResourceArray + * Remember that an object is owner by a ReourceOwner * - * Caller must have previously done ResourceArrayEnlarge() + * Caller must have previously done ResourceOwnerEnlarge() */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 idx; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Append to linear array. */ - idx = resarr->nitems; - } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Remove a resource from ResourceArray + * Forget that an object is owned by a ResourceOwner * - * Returns true on success, false if resource was not found. + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. * - * Note: if same resource ID appears more than once, one instance is removed. + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 i, - idx, - lastidx = resarr->lastidx; + idx; - Assert(value != resarr->invalidval); +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) - { - if (resarr->itemsarr[i] == value) - { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + +#ifdef RESOWNER_STATS + narray_lookups++; +#endif + + return; } } - else + + /* Search hash */ + if (owner->nhash > 0) { - uint32 mask = resarr->capacity - 1; + uint32 mask = owner->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) + for (i = 0; i < owner->capacity; i++) { - if (resarr->itemsarr[idx] == value) + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; } idx = (idx + 1) & mask; } } - return false; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Call the ReleaseResource callback on entries with given 'phase'. */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +static void +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - if (resarr->nitems == 0) - return false; + bool found; + int capacity; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else + /* First handle all the entries in the array. */ + do { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; - - for (;;) + found = false; + for (int i = 0; i < owner->narr; i++) { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } } - } - *value = resarr->itemsarr[resarr->lastidx]; - return true; -} + /* + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. + */ + } while (found && owner->narr > 0); -/* - * Trash a ResourceArray (we don't care about its state after this) - */ -static void -ResourceArrayFree(ResourceArray *resarr) -{ - if (resarr->itemsarr) - pfree(resarr->itemsarr); + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) + { + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } + } + + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } @@ -437,17 +441,10 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name) parent->firstchild = owner; } - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif return owner; } @@ -486,6 +483,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +503,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +518,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +533,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +577,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +600,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - ReleaseCachedPlan(res, true); + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) + { + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + CurrentResourceOwner = save; } @@ -730,19 +652,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +676,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +830,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +881,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..1ed1580c644 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..281f85586fa 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,21 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +static inline void +ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} +static inline void +ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------3CC8131494C3F8E34BADAF02-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v4 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1212 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 803 insertions(+), 1052 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..34896b303ac 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all objects the same, but to register a +new kind of an object with it, you need to fill in a few callback +functions, see ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..99240679e09 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ - + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, true); + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +841,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +892,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------7349B9D66ED595EF3B1030B4-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v5 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1221 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 811 insertions(+), 1057 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..c67ca603526 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resource the same, and extensions +can define a new kind of resource by defining a new ResourceOwnerFuncs +instance with a few callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..75fda4fcc13 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. + * Remember that an object is owner by a ReourceOwner * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; - - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +746,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +840,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +891,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------1104342304EC6341F5D7C60C Content-Type: text/x-patch; charset=UTF-8; name="0001-resownerbench.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-resownerbench.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v2 1/1] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 43 +- src/backend/storage/buffer/localbuf.c | 2 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 98 ++- src/backend/utils/cache/plancache.c | 50 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/cache/syscache.c | 2 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1139 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 15 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 21 files changed, 795 insertions(+), 1020 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 30c30cf3a2e..c099f04f551 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 5ca3f922fed..c44080b9742 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 9c4fc75f656..d01a315b20f 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c5e87071517..7c6f174c489 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -198,6 +198,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -727,7 +739,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); isExtend = (blockNum == P_NEW); @@ -1846,7 +1858,7 @@ BufferSync(int flags) WritebackContext wb_context; /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Unless this is a shutdown checkpoint or we have been explicitly told, @@ -2323,7 +2335,7 @@ BgBufferSync(WritebackContext *wb_context) */ /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); num_to_scan = bufs_to_lap; num_written = 0; @@ -3300,7 +3312,7 @@ FlushRelationBuffers(Relation rel) } /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3373,7 +3385,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3452,7 +3464,7 @@ FlushDatabaseBuffers(Oid dbid) BufferDesc *bufHdr; /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3550,7 +3562,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4584,3 +4596,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index cd3475e9e1d..4eeed7d66f9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index f07b5325aa5..5d85bd8d1a7 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -96,7 +96,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -339,6 +339,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1429,7 +1447,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1611,7 +1629,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1741,7 +1759,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1779,7 +1797,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3635,3 +3653,19 @@ data_sync_elevel(int elevel) { return data_sync_retry ? elevel : PANIC; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index dffbd8e82a2..c665bc6cecf 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 53472dd21ec..25e025f4f57 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 3613ae5f44d..c9ebf9effe1 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,66 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* support for catcache refcount management */ +static inline void +ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs); +} +static inline void +ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs); +} + +static inline void +ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs); +} + +static inline void +ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs); +} + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1331,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1432,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1644,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1607,7 +1668,7 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ctlist = NIL; @@ -2062,14 +2123,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) { + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) +{ + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2083,9 +2149,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) +{ + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) { + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 50d6ad28b4c..8bce9d16ed5 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +static inline void +ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} +static inline void +ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1254,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1417,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1476,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2230,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 3bd5e180425..534e33a23c3 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2065,6 +2066,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2076,10 +2089,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2092,7 +2105,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6404,3 +6417,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index 809b27a038f..a3c4063e655 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -982,7 +982,7 @@ static const struct cachedesc cacheinfo[] = { } }; -static CatCache *SysCache[SysCacheSize]; + CatCache *SysCache[SysCacheSize]; static bool CacheInitialized = false; diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..890db7d1e66 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context references. +ResourceOwner treates all objects the same, but to register a new kind of +an object with it, you need to fill in a few callback functions, see +ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callabacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 546ad8d1c5f..9806793e03b 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) - -/* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. + * ResourceElem represents a reference associated with a resource owner. * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. - * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,45 +153,34 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ - -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* @@ -207,205 +189,227 @@ ResourceArrayInit(ResourceArray *resarr, Datum invalidval) * This is separate from actually inserting a resource because if we run out * of memory, it's critical to do so *before* acquiring the resource. */ -static void -ResourceArrayEnlarge(ResourceArray *resarr) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) + if (owner->narr < RESOWNER_ARRAY_SIZE) return; /* no work needed */ - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; - - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; - - if (olditemsarr != NULL) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. - */ - for (i = 0; i < oldcap; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) + { + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); + } + + /* And release old hash table. */ + pfree(oldhash); } + } - /* And release old array. */ - pfree(olditemsarr); + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - Assert(resarr->nitems < resarr->maxitems); + Assert(owner->nhash < owner->grow_at); } /* - * Add a resource to ResourceArray + * Remember that an object is owner by a ReourceOwner * - * Caller must have previously done ResourceArrayEnlarge() + * Caller must have previously done ResourceOwnerEnlarge() */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 idx; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Append to linear array. */ - idx = resarr->nitems; - } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Remove a resource from ResourceArray + * Forget that an object is owned by a ResourceOwner * - * Returns true on success, false if resource was not found. + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. * - * Note: if same resource ID appears more than once, one instance is removed. + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 i, - idx, - lastidx = resarr->lastidx; + idx; - Assert(value != resarr->invalidval); +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) - { - if (resarr->itemsarr[i] == value) - { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + +#ifdef RESOWNER_STATS + narray_lookups++; +#endif + + return; } } - else + + /* Search hash */ + if (owner->nhash > 0) { - uint32 mask = resarr->capacity - 1; + uint32 mask = owner->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) + for (i = 0; i < owner->capacity; i++) { - if (resarr->itemsarr[idx] == value) + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; } idx = (idx + 1) & mask; } } - return false; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Call the ReleaseResource callback on entries with given 'phase'. */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +static void +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - if (resarr->nitems == 0) - return false; + bool found; + int capacity; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else + /* First handle all the entries in the array. */ + do { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; - - for (;;) + found = false; + for (int i = 0; i < owner->narr; i++) { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } } - } - *value = resarr->itemsarr[resarr->lastidx]; - return true; -} + /* + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. + */ + } while (found && owner->narr > 0); -/* - * Trash a ResourceArray (we don't care about its state after this) - */ -static void -ResourceArrayFree(ResourceArray *resarr) -{ - if (resarr->itemsarr) - pfree(resarr->itemsarr); + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) + { + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } + } + + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } @@ -437,17 +441,10 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name) parent->firstchild = owner; } - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif return owner; } @@ -486,6 +483,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +503,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +518,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +533,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +577,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +600,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - ReleaseCachedPlan(res, true); + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) + { + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + CurrentResourceOwner = save; } @@ -730,19 +652,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +676,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +830,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +881,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 8c41483e87c..b3ea3ce7650 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 118651c4153..d3c610a86b0 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -57,6 +56,21 @@ typedef struct pg_cryptohash_state #endif } pg_cryptohash_state; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -85,7 +99,7 @@ pg_cryptohash_create(pg_cryptohash_type type) ctx->type = type; #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif /* @@ -110,8 +124,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND state->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -221,8 +235,8 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(state->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(state->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(state->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(state, sizeof(pg_cryptohash_state)); @@ -230,3 +244,22 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) FREE(state); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index 3377fa56768..c4aa7d7e31e 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,21 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +static inline void +ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} +static inline void +ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index f4aa316604e..dbb10dfdd16 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 4901568553c..2e0d5006719 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 878f39ccf14..7fa1c1e50ac 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c373788bc13..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.20.1 --------------6CCFD766DA3387BD841083BC-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v3 1/1] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 43 +- src/backend/storage/buffer/localbuf.c | 2 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 98 ++- src/backend/utils/cache/plancache.c | 50 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/cache/syscache.c | 2 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1139 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 15 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 21 files changed, 795 insertions(+), 1020 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c46b8abad12..8b77fb68941 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -739,7 +751,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); isExtend = (blockNum == P_NEW); @@ -1858,7 +1870,7 @@ BufferSync(int flags) WritebackContext wb_context; /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Unless this is a shutdown checkpoint or we have been explicitly told, @@ -2335,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context) */ /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); num_to_scan = bufs_to_lap; num_written = 0; @@ -3488,7 +3500,7 @@ FlushRelationBuffers(Relation rel) } /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3561,7 +3573,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3640,7 +3652,7 @@ FlushDatabaseBuffers(Oid dbid) BufferDesc *bufHdr; /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3738,7 +3750,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4802,3 +4814,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 04b3558ea33..811e0e3a45e 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index 931ed679307..5d2394e6780 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -96,7 +96,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -339,6 +339,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1429,7 +1447,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1611,7 +1629,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1741,7 +1759,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1779,7 +1797,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3635,3 +3653,19 @@ data_sync_elevel(int elevel) { return data_sync_retry ? elevel : PANIC; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index fa2b49c676e..24751c0d3f4 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,66 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* support for catcache refcount management */ +static inline void +ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs); +} +static inline void +ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs); +} + +static inline void +ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs); +} + +static inline void +ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs); +} + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1331,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1432,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1644,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1607,7 +1668,7 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ctlist = NIL; @@ -2062,14 +2123,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) { + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) +{ + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2083,9 +2149,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) +{ + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) { + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..484f14f5524 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +static inline void +ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} +static inline void +ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1254,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1417,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1476,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2230,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index e4dc4ee34ee..d6fe6a5bd5a 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -994,7 +994,7 @@ static const struct cachedesc cacheinfo[] = { } }; -static CatCache *SysCache[SysCacheSize]; + CatCache *SysCache[SysCacheSize]; static bool CacheInitialized = false; diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..890db7d1e66 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context references. +ResourceOwner treates all objects the same, but to register a new kind of +an object with it, you need to fill in a few callback functions, see +ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callabacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..c49758cb20d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) - -/* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. + * ResourceElem represents a reference associated with a resource owner. * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. - * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,45 +153,34 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ - -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* @@ -207,205 +189,227 @@ ResourceArrayInit(ResourceArray *resarr, Datum invalidval) * This is separate from actually inserting a resource because if we run out * of memory, it's critical to do so *before* acquiring the resource. */ -static void -ResourceArrayEnlarge(ResourceArray *resarr) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) + if (owner->narr < RESOWNER_ARRAY_SIZE) return; /* no work needed */ - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; - - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; - - if (olditemsarr != NULL) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. - */ - for (i = 0; i < oldcap; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) + { + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); + } + + /* And release old hash table. */ + pfree(oldhash); } + } - /* And release old array. */ - pfree(olditemsarr); + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - Assert(resarr->nitems < resarr->maxitems); + Assert(owner->nhash < owner->grow_at); } /* - * Add a resource to ResourceArray + * Remember that an object is owner by a ReourceOwner * - * Caller must have previously done ResourceArrayEnlarge() + * Caller must have previously done ResourceOwnerEnlarge() */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 idx; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Append to linear array. */ - idx = resarr->nitems; - } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Remove a resource from ResourceArray + * Forget that an object is owned by a ResourceOwner * - * Returns true on success, false if resource was not found. + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. * - * Note: if same resource ID appears more than once, one instance is removed. + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 i, - idx, - lastidx = resarr->lastidx; + idx; - Assert(value != resarr->invalidval); +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) - { - if (resarr->itemsarr[i] == value) - { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + +#ifdef RESOWNER_STATS + narray_lookups++; +#endif + + return; } } - else + + /* Search hash */ + if (owner->nhash > 0) { - uint32 mask = resarr->capacity - 1; + uint32 mask = owner->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) + for (i = 0; i < owner->capacity; i++) { - if (resarr->itemsarr[idx] == value) + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; } idx = (idx + 1) & mask; } } - return false; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Call the ReleaseResource callback on entries with given 'phase'. */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +static void +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - if (resarr->nitems == 0) - return false; + bool found; + int capacity; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else + /* First handle all the entries in the array. */ + do { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; - - for (;;) + found = false; + for (int i = 0; i < owner->narr; i++) { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } } - } - *value = resarr->itemsarr[resarr->lastidx]; - return true; -} + /* + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. + */ + } while (found && owner->narr > 0); -/* - * Trash a ResourceArray (we don't care about its state after this) - */ -static void -ResourceArrayFree(ResourceArray *resarr) -{ - if (resarr->itemsarr) - pfree(resarr->itemsarr); + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) + { + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } + } + + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } @@ -437,17 +441,10 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name) parent->firstchild = owner; } - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif return owner; } @@ -486,6 +483,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +503,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +518,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +533,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +577,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +600,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - ReleaseCachedPlan(res, true); + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) + { + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + CurrentResourceOwner = save; } @@ -730,19 +652,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +676,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +830,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +881,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..1ed1580c644 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..281f85586fa 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,21 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +static inline void +ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} +static inline void +ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------3CC8131494C3F8E34BADAF02-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v3 1/1] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 43 +- src/backend/storage/buffer/localbuf.c | 2 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 98 ++- src/backend/utils/cache/plancache.c | 50 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/cache/syscache.c | 2 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1139 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 15 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 21 files changed, 795 insertions(+), 1020 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c46b8abad12..8b77fb68941 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -739,7 +751,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); isExtend = (blockNum == P_NEW); @@ -1858,7 +1870,7 @@ BufferSync(int flags) WritebackContext wb_context; /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Unless this is a shutdown checkpoint or we have been explicitly told, @@ -2335,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context) */ /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); num_to_scan = bufs_to_lap; num_written = 0; @@ -3488,7 +3500,7 @@ FlushRelationBuffers(Relation rel) } /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3561,7 +3573,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3640,7 +3652,7 @@ FlushDatabaseBuffers(Oid dbid) BufferDesc *bufHdr; /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3738,7 +3750,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4802,3 +4814,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 04b3558ea33..811e0e3a45e 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index 931ed679307..5d2394e6780 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -96,7 +96,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -339,6 +339,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1429,7 +1447,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1611,7 +1629,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1741,7 +1759,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1779,7 +1797,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3635,3 +3653,19 @@ data_sync_elevel(int elevel) { return data_sync_retry ? elevel : PANIC; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index fa2b49c676e..24751c0d3f4 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,66 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* support for catcache refcount management */ +static inline void +ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs); +} +static inline void +ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs); +} + +static inline void +ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs); +} + +static inline void +ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs); +} + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1331,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1432,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1644,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1607,7 +1668,7 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ctlist = NIL; @@ -2062,14 +2123,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) { + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) +{ + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2083,9 +2149,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) +{ + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) { + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..484f14f5524 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +static inline void +ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} +static inline void +ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1254,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1417,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1476,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2230,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index e4dc4ee34ee..d6fe6a5bd5a 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -994,7 +994,7 @@ static const struct cachedesc cacheinfo[] = { } }; -static CatCache *SysCache[SysCacheSize]; + CatCache *SysCache[SysCacheSize]; static bool CacheInitialized = false; diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..890db7d1e66 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context references. +ResourceOwner treates all objects the same, but to register a new kind of +an object with it, you need to fill in a few callback functions, see +ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callabacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..c49758cb20d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) - -/* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. + * ResourceElem represents a reference associated with a resource owner. * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. - * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,45 +153,34 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ - -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* @@ -207,205 +189,227 @@ ResourceArrayInit(ResourceArray *resarr, Datum invalidval) * This is separate from actually inserting a resource because if we run out * of memory, it's critical to do so *before* acquiring the resource. */ -static void -ResourceArrayEnlarge(ResourceArray *resarr) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) + if (owner->narr < RESOWNER_ARRAY_SIZE) return; /* no work needed */ - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; - - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; - - if (olditemsarr != NULL) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. - */ - for (i = 0; i < oldcap; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) + { + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); + } + + /* And release old hash table. */ + pfree(oldhash); } + } - /* And release old array. */ - pfree(olditemsarr); + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - Assert(resarr->nitems < resarr->maxitems); + Assert(owner->nhash < owner->grow_at); } /* - * Add a resource to ResourceArray + * Remember that an object is owner by a ReourceOwner * - * Caller must have previously done ResourceArrayEnlarge() + * Caller must have previously done ResourceOwnerEnlarge() */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 idx; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Append to linear array. */ - idx = resarr->nitems; - } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Remove a resource from ResourceArray + * Forget that an object is owned by a ResourceOwner * - * Returns true on success, false if resource was not found. + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. * - * Note: if same resource ID appears more than once, one instance is removed. + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 i, - idx, - lastidx = resarr->lastidx; + idx; - Assert(value != resarr->invalidval); +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) - { - if (resarr->itemsarr[i] == value) - { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + +#ifdef RESOWNER_STATS + narray_lookups++; +#endif + + return; } } - else + + /* Search hash */ + if (owner->nhash > 0) { - uint32 mask = resarr->capacity - 1; + uint32 mask = owner->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) + for (i = 0; i < owner->capacity; i++) { - if (resarr->itemsarr[idx] == value) + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; } idx = (idx + 1) & mask; } } - return false; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Call the ReleaseResource callback on entries with given 'phase'. */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +static void +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - if (resarr->nitems == 0) - return false; + bool found; + int capacity; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else + /* First handle all the entries in the array. */ + do { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; - - for (;;) + found = false; + for (int i = 0; i < owner->narr; i++) { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } } - } - *value = resarr->itemsarr[resarr->lastidx]; - return true; -} + /* + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. + */ + } while (found && owner->narr > 0); -/* - * Trash a ResourceArray (we don't care about its state after this) - */ -static void -ResourceArrayFree(ResourceArray *resarr) -{ - if (resarr->itemsarr) - pfree(resarr->itemsarr); + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) + { + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } + } + + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } @@ -437,17 +441,10 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name) parent->firstchild = owner; } - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif return owner; } @@ -486,6 +483,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +503,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +518,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +533,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +577,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +600,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - ReleaseCachedPlan(res, true); + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) + { + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + CurrentResourceOwner = save; } @@ -730,19 +652,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +676,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +830,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +881,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..1ed1580c644 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..281f85586fa 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,21 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +static inline void +ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} +static inline void +ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------3CC8131494C3F8E34BADAF02-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v4 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1212 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 803 insertions(+), 1052 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..34896b303ac 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all objects the same, but to register a +new kind of an object with it, you need to fill in a few callback +functions, see ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..99240679e09 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ - + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, true); + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +841,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +892,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------7349B9D66ED595EF3B1030B4-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v5 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1221 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 811 insertions(+), 1057 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..c67ca603526 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resource the same, and extensions +can define a new kind of resource by defining a new ResourceOwnerFuncs +instance with a few callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..75fda4fcc13 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. + * Remember that an object is owner by a ReourceOwner * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; - - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +746,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +840,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +891,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------1104342304EC6341F5D7C60C Content-Type: text/x-patch; charset=UTF-8; name="0001-resownerbench.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-resownerbench.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v3 1/1] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 43 +- src/backend/storage/buffer/localbuf.c | 2 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 98 ++- src/backend/utils/cache/plancache.c | 50 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/cache/syscache.c | 2 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1139 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 15 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 21 files changed, 795 insertions(+), 1020 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c46b8abad12..8b77fb68941 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -739,7 +751,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); isExtend = (blockNum == P_NEW); @@ -1858,7 +1870,7 @@ BufferSync(int flags) WritebackContext wb_context; /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Unless this is a shutdown checkpoint or we have been explicitly told, @@ -2335,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context) */ /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); num_to_scan = bufs_to_lap; num_written = 0; @@ -3488,7 +3500,7 @@ FlushRelationBuffers(Relation rel) } /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3561,7 +3573,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3640,7 +3652,7 @@ FlushDatabaseBuffers(Oid dbid) BufferDesc *bufHdr; /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3738,7 +3750,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4802,3 +4814,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 04b3558ea33..811e0e3a45e 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index 931ed679307..5d2394e6780 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -96,7 +96,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -339,6 +339,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1429,7 +1447,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1611,7 +1629,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1741,7 +1759,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1779,7 +1797,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3635,3 +3653,19 @@ data_sync_elevel(int elevel) { return data_sync_retry ? elevel : PANIC; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index fa2b49c676e..24751c0d3f4 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,66 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* support for catcache refcount management */ +static inline void +ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs); +} +static inline void +ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs); +} + +static inline void +ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs); +} + +static inline void +ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs); +} + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1331,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1432,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1644,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1607,7 +1668,7 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ctlist = NIL; @@ -2062,14 +2123,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) { + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) +{ + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2083,9 +2149,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) +{ + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) { + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..484f14f5524 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +static inline void +ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} +static inline void +ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1254,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1417,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1476,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2230,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index e4dc4ee34ee..d6fe6a5bd5a 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -994,7 +994,7 @@ static const struct cachedesc cacheinfo[] = { } }; -static CatCache *SysCache[SysCacheSize]; + CatCache *SysCache[SysCacheSize]; static bool CacheInitialized = false; diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..890db7d1e66 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context references. +ResourceOwner treates all objects the same, but to register a new kind of +an object with it, you need to fill in a few callback functions, see +ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callabacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..c49758cb20d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) - -/* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. + * ResourceElem represents a reference associated with a resource owner. * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. - * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,45 +153,34 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ - -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* @@ -207,205 +189,227 @@ ResourceArrayInit(ResourceArray *resarr, Datum invalidval) * This is separate from actually inserting a resource because if we run out * of memory, it's critical to do so *before* acquiring the resource. */ -static void -ResourceArrayEnlarge(ResourceArray *resarr) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) + if (owner->narr < RESOWNER_ARRAY_SIZE) return; /* no work needed */ - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; - - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; - - if (olditemsarr != NULL) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. - */ - for (i = 0; i < oldcap; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) + { + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); + } + + /* And release old hash table. */ + pfree(oldhash); } + } - /* And release old array. */ - pfree(olditemsarr); + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - Assert(resarr->nitems < resarr->maxitems); + Assert(owner->nhash < owner->grow_at); } /* - * Add a resource to ResourceArray + * Remember that an object is owner by a ReourceOwner * - * Caller must have previously done ResourceArrayEnlarge() + * Caller must have previously done ResourceOwnerEnlarge() */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 idx; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Append to linear array. */ - idx = resarr->nitems; - } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Remove a resource from ResourceArray + * Forget that an object is owned by a ResourceOwner * - * Returns true on success, false if resource was not found. + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. * - * Note: if same resource ID appears more than once, one instance is removed. + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 i, - idx, - lastidx = resarr->lastidx; + idx; - Assert(value != resarr->invalidval); +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) - { - if (resarr->itemsarr[i] == value) - { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + +#ifdef RESOWNER_STATS + narray_lookups++; +#endif + + return; } } - else + + /* Search hash */ + if (owner->nhash > 0) { - uint32 mask = resarr->capacity - 1; + uint32 mask = owner->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) + for (i = 0; i < owner->capacity; i++) { - if (resarr->itemsarr[idx] == value) + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; } idx = (idx + 1) & mask; } } - return false; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Call the ReleaseResource callback on entries with given 'phase'. */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +static void +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - if (resarr->nitems == 0) - return false; + bool found; + int capacity; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else + /* First handle all the entries in the array. */ + do { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; - - for (;;) + found = false; + for (int i = 0; i < owner->narr; i++) { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } } - } - *value = resarr->itemsarr[resarr->lastidx]; - return true; -} + /* + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. + */ + } while (found && owner->narr > 0); -/* - * Trash a ResourceArray (we don't care about its state after this) - */ -static void -ResourceArrayFree(ResourceArray *resarr) -{ - if (resarr->itemsarr) - pfree(resarr->itemsarr); + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) + { + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } + } + + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } @@ -437,17 +441,10 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name) parent->firstchild = owner; } - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif return owner; } @@ -486,6 +483,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +503,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +518,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +533,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +577,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +600,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - ReleaseCachedPlan(res, true); + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) + { + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + CurrentResourceOwner = save; } @@ -730,19 +652,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +676,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +830,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +881,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..1ed1580c644 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..281f85586fa 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,21 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +static inline void +ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} +static inline void +ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------3CC8131494C3F8E34BADAF02-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v4 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1212 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 803 insertions(+), 1052 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..34896b303ac 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all objects the same, but to register a +new kind of an object with it, you need to fill in a few callback +functions, see ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..99240679e09 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ - + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, true); + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +841,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +892,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------7349B9D66ED595EF3B1030B4-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v5 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1221 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 811 insertions(+), 1057 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..c67ca603526 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resource the same, and extensions +can define a new kind of resource by defining a new ResourceOwnerFuncs +instance with a few callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..75fda4fcc13 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. + * Remember that an object is owner by a ReourceOwner * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; - - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +746,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +840,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +891,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------1104342304EC6341F5D7C60C Content-Type: text/x-patch; charset=UTF-8; name="0001-resownerbench.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-resownerbench.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v3 1/1] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 43 +- src/backend/storage/buffer/localbuf.c | 2 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 98 ++- src/backend/utils/cache/plancache.c | 50 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/cache/syscache.c | 2 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1139 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 15 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 21 files changed, 795 insertions(+), 1020 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c46b8abad12..8b77fb68941 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -739,7 +751,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); isExtend = (blockNum == P_NEW); @@ -1858,7 +1870,7 @@ BufferSync(int flags) WritebackContext wb_context; /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Unless this is a shutdown checkpoint or we have been explicitly told, @@ -2335,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context) */ /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); num_to_scan = bufs_to_lap; num_written = 0; @@ -3488,7 +3500,7 @@ FlushRelationBuffers(Relation rel) } /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3561,7 +3573,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3640,7 +3652,7 @@ FlushDatabaseBuffers(Oid dbid) BufferDesc *bufHdr; /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3738,7 +3750,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4802,3 +4814,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 04b3558ea33..811e0e3a45e 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index 931ed679307..5d2394e6780 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -96,7 +96,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -339,6 +339,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1429,7 +1447,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1611,7 +1629,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1741,7 +1759,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1779,7 +1797,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3635,3 +3653,19 @@ data_sync_elevel(int elevel) { return data_sync_retry ? elevel : PANIC; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index fa2b49c676e..24751c0d3f4 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,66 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* support for catcache refcount management */ +static inline void +ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs); +} +static inline void +ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs); +} + +static inline void +ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs); +} + +static inline void +ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs); +} + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1331,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1432,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1644,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1607,7 +1668,7 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ctlist = NIL; @@ -2062,14 +2123,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) { + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) +{ + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2083,9 +2149,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) +{ + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) { + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..484f14f5524 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +static inline void +ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} +static inline void +ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1254,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1417,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1476,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2230,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index e4dc4ee34ee..d6fe6a5bd5a 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -994,7 +994,7 @@ static const struct cachedesc cacheinfo[] = { } }; -static CatCache *SysCache[SysCacheSize]; + CatCache *SysCache[SysCacheSize]; static bool CacheInitialized = false; diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..890db7d1e66 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context references. +ResourceOwner treates all objects the same, but to register a new kind of +an object with it, you need to fill in a few callback functions, see +ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callabacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..c49758cb20d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) - -/* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. + * ResourceElem represents a reference associated with a resource owner. * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. - * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,45 +153,34 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ - -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* @@ -207,205 +189,227 @@ ResourceArrayInit(ResourceArray *resarr, Datum invalidval) * This is separate from actually inserting a resource because if we run out * of memory, it's critical to do so *before* acquiring the resource. */ -static void -ResourceArrayEnlarge(ResourceArray *resarr) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) + if (owner->narr < RESOWNER_ARRAY_SIZE) return; /* no work needed */ - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; - - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; - - if (olditemsarr != NULL) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. - */ - for (i = 0; i < oldcap; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) + { + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); + } + + /* And release old hash table. */ + pfree(oldhash); } + } - /* And release old array. */ - pfree(olditemsarr); + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - Assert(resarr->nitems < resarr->maxitems); + Assert(owner->nhash < owner->grow_at); } /* - * Add a resource to ResourceArray + * Remember that an object is owner by a ReourceOwner * - * Caller must have previously done ResourceArrayEnlarge() + * Caller must have previously done ResourceOwnerEnlarge() */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 idx; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Append to linear array. */ - idx = resarr->nitems; - } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Remove a resource from ResourceArray + * Forget that an object is owned by a ResourceOwner * - * Returns true on success, false if resource was not found. + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. * - * Note: if same resource ID appears more than once, one instance is removed. + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 i, - idx, - lastidx = resarr->lastidx; + idx; - Assert(value != resarr->invalidval); +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) - { - if (resarr->itemsarr[i] == value) - { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + +#ifdef RESOWNER_STATS + narray_lookups++; +#endif + + return; } } - else + + /* Search hash */ + if (owner->nhash > 0) { - uint32 mask = resarr->capacity - 1; + uint32 mask = owner->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) + for (i = 0; i < owner->capacity; i++) { - if (resarr->itemsarr[idx] == value) + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; } idx = (idx + 1) & mask; } } - return false; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Call the ReleaseResource callback on entries with given 'phase'. */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +static void +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - if (resarr->nitems == 0) - return false; + bool found; + int capacity; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else + /* First handle all the entries in the array. */ + do { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; - - for (;;) + found = false; + for (int i = 0; i < owner->narr; i++) { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } } - } - *value = resarr->itemsarr[resarr->lastidx]; - return true; -} + /* + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. + */ + } while (found && owner->narr > 0); -/* - * Trash a ResourceArray (we don't care about its state after this) - */ -static void -ResourceArrayFree(ResourceArray *resarr) -{ - if (resarr->itemsarr) - pfree(resarr->itemsarr); + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) + { + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } + } + + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } @@ -437,17 +441,10 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name) parent->firstchild = owner; } - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif return owner; } @@ -486,6 +483,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +503,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +518,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +533,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +577,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +600,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - ReleaseCachedPlan(res, true); + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) + { + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + CurrentResourceOwner = save; } @@ -730,19 +652,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +676,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +830,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +881,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..1ed1580c644 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..281f85586fa 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,21 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +static inline void +ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} +static inline void +ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------3CC8131494C3F8E34BADAF02-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v4 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1212 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 803 insertions(+), 1052 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..34896b303ac 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all objects the same, but to register a +new kind of an object with it, you need to fill in a few callback +functions, see ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..99240679e09 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ - + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, true); + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +841,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +892,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------7349B9D66ED595EF3B1030B4-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v5 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1221 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 811 insertions(+), 1057 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..c67ca603526 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resource the same, and extensions +can define a new kind of resource by defining a new ResourceOwnerFuncs +instance with a few callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..75fda4fcc13 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. + * Remember that an object is owner by a ReourceOwner * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; - - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +746,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +840,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +891,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------1104342304EC6341F5D7C60C Content-Type: text/x-patch; charset=UTF-8; name="0001-resownerbench.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-resownerbench.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v3 1/1] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 43 +- src/backend/storage/buffer/localbuf.c | 2 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 98 ++- src/backend/utils/cache/plancache.c | 50 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/cache/syscache.c | 2 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1139 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 15 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 21 files changed, 795 insertions(+), 1020 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c46b8abad12..8b77fb68941 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -739,7 +751,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); isExtend = (blockNum == P_NEW); @@ -1858,7 +1870,7 @@ BufferSync(int flags) WritebackContext wb_context; /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Unless this is a shutdown checkpoint or we have been explicitly told, @@ -2335,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context) */ /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); num_to_scan = bufs_to_lap; num_written = 0; @@ -3488,7 +3500,7 @@ FlushRelationBuffers(Relation rel) } /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3561,7 +3573,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3640,7 +3652,7 @@ FlushDatabaseBuffers(Oid dbid) BufferDesc *bufHdr; /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3738,7 +3750,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4802,3 +4814,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 04b3558ea33..811e0e3a45e 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index 931ed679307..5d2394e6780 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -96,7 +96,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -339,6 +339,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1429,7 +1447,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1611,7 +1629,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1741,7 +1759,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1779,7 +1797,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3635,3 +3653,19 @@ data_sync_elevel(int elevel) { return data_sync_retry ? elevel : PANIC; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index fa2b49c676e..24751c0d3f4 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,66 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* support for catcache refcount management */ +static inline void +ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs); +} +static inline void +ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs); +} + +static inline void +ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs); +} + +static inline void +ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs); +} + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1331,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1432,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1644,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1607,7 +1668,7 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ctlist = NIL; @@ -2062,14 +2123,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) { + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) +{ + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2083,9 +2149,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) +{ + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) { + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..484f14f5524 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +static inline void +ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} +static inline void +ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1254,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1417,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1476,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2230,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index e4dc4ee34ee..d6fe6a5bd5a 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -994,7 +994,7 @@ static const struct cachedesc cacheinfo[] = { } }; -static CatCache *SysCache[SysCacheSize]; + CatCache *SysCache[SysCacheSize]; static bool CacheInitialized = false; diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..890db7d1e66 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context references. +ResourceOwner treates all objects the same, but to register a new kind of +an object with it, you need to fill in a few callback functions, see +ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callabacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..c49758cb20d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) - -/* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. + * ResourceElem represents a reference associated with a resource owner. * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. - * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,45 +153,34 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ - -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* @@ -207,205 +189,227 @@ ResourceArrayInit(ResourceArray *resarr, Datum invalidval) * This is separate from actually inserting a resource because if we run out * of memory, it's critical to do so *before* acquiring the resource. */ -static void -ResourceArrayEnlarge(ResourceArray *resarr) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) + if (owner->narr < RESOWNER_ARRAY_SIZE) return; /* no work needed */ - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; - - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; - - if (olditemsarr != NULL) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. - */ - for (i = 0; i < oldcap; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) + { + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); + } + + /* And release old hash table. */ + pfree(oldhash); } + } - /* And release old array. */ - pfree(olditemsarr); + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - Assert(resarr->nitems < resarr->maxitems); + Assert(owner->nhash < owner->grow_at); } /* - * Add a resource to ResourceArray + * Remember that an object is owner by a ReourceOwner * - * Caller must have previously done ResourceArrayEnlarge() + * Caller must have previously done ResourceOwnerEnlarge() */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 idx; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Append to linear array. */ - idx = resarr->nitems; - } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Remove a resource from ResourceArray + * Forget that an object is owned by a ResourceOwner * - * Returns true on success, false if resource was not found. + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. * - * Note: if same resource ID appears more than once, one instance is removed. + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 i, - idx, - lastidx = resarr->lastidx; + idx; - Assert(value != resarr->invalidval); +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) - { - if (resarr->itemsarr[i] == value) - { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + +#ifdef RESOWNER_STATS + narray_lookups++; +#endif + + return; } } - else + + /* Search hash */ + if (owner->nhash > 0) { - uint32 mask = resarr->capacity - 1; + uint32 mask = owner->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) + for (i = 0; i < owner->capacity; i++) { - if (resarr->itemsarr[idx] == value) + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; } idx = (idx + 1) & mask; } } - return false; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Call the ReleaseResource callback on entries with given 'phase'. */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +static void +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - if (resarr->nitems == 0) - return false; + bool found; + int capacity; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else + /* First handle all the entries in the array. */ + do { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; - - for (;;) + found = false; + for (int i = 0; i < owner->narr; i++) { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } } - } - *value = resarr->itemsarr[resarr->lastidx]; - return true; -} + /* + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. + */ + } while (found && owner->narr > 0); -/* - * Trash a ResourceArray (we don't care about its state after this) - */ -static void -ResourceArrayFree(ResourceArray *resarr) -{ - if (resarr->itemsarr) - pfree(resarr->itemsarr); + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) + { + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } + } + + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } @@ -437,17 +441,10 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name) parent->firstchild = owner; } - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif return owner; } @@ -486,6 +483,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +503,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +518,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +533,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +577,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +600,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - ReleaseCachedPlan(res, true); + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) + { + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + CurrentResourceOwner = save; } @@ -730,19 +652,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +676,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +830,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +881,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..1ed1580c644 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..281f85586fa 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,21 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +static inline void +ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} +static inline void +ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------3CC8131494C3F8E34BADAF02-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v4 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1212 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 803 insertions(+), 1052 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..34896b303ac 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all objects the same, but to register a +new kind of an object with it, you need to fill in a few callback +functions, see ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..99240679e09 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ - + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, true); + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +841,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +892,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------7349B9D66ED595EF3B1030B4-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v5 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1221 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 811 insertions(+), 1057 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..c67ca603526 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resource the same, and extensions +can define a new kind of resource by defining a new ResourceOwnerFuncs +instance with a few callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..75fda4fcc13 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. + * Remember that an object is owner by a ReourceOwner * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; - - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +746,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +840,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +891,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------1104342304EC6341F5D7C60C Content-Type: text/x-patch; charset=UTF-8; name="0001-resownerbench.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-resownerbench.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v3 1/1] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 43 +- src/backend/storage/buffer/localbuf.c | 2 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 98 ++- src/backend/utils/cache/plancache.c | 50 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/cache/syscache.c | 2 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1139 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 15 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 21 files changed, 795 insertions(+), 1020 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c46b8abad12..8b77fb68941 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -739,7 +751,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); isExtend = (blockNum == P_NEW); @@ -1858,7 +1870,7 @@ BufferSync(int flags) WritebackContext wb_context; /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Unless this is a shutdown checkpoint or we have been explicitly told, @@ -2335,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context) */ /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); num_to_scan = bufs_to_lap; num_written = 0; @@ -3488,7 +3500,7 @@ FlushRelationBuffers(Relation rel) } /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3561,7 +3573,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3640,7 +3652,7 @@ FlushDatabaseBuffers(Oid dbid) BufferDesc *bufHdr; /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3738,7 +3750,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4802,3 +4814,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 04b3558ea33..811e0e3a45e 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index 931ed679307..5d2394e6780 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -96,7 +96,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -339,6 +339,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1429,7 +1447,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1611,7 +1629,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1741,7 +1759,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1779,7 +1797,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3635,3 +3653,19 @@ data_sync_elevel(int elevel) { return data_sync_retry ? elevel : PANIC; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index fa2b49c676e..24751c0d3f4 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,66 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* support for catcache refcount management */ +static inline void +ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs); +} +static inline void +ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs); +} + +static inline void +ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs); +} + +static inline void +ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs); +} + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1331,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1432,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1644,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1607,7 +1668,7 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ctlist = NIL; @@ -2062,14 +2123,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) { + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) +{ + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2083,9 +2149,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) +{ + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) { + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..484f14f5524 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +static inline void +ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} +static inline void +ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1254,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1417,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1476,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2230,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index e4dc4ee34ee..d6fe6a5bd5a 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -994,7 +994,7 @@ static const struct cachedesc cacheinfo[] = { } }; -static CatCache *SysCache[SysCacheSize]; + CatCache *SysCache[SysCacheSize]; static bool CacheInitialized = false; diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..890db7d1e66 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context references. +ResourceOwner treates all objects the same, but to register a new kind of +an object with it, you need to fill in a few callback functions, see +ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callabacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..c49758cb20d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) - -/* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. + * ResourceElem represents a reference associated with a resource owner. * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. - * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,45 +153,34 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ - -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* @@ -207,205 +189,227 @@ ResourceArrayInit(ResourceArray *resarr, Datum invalidval) * This is separate from actually inserting a resource because if we run out * of memory, it's critical to do so *before* acquiring the resource. */ -static void -ResourceArrayEnlarge(ResourceArray *resarr) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) + if (owner->narr < RESOWNER_ARRAY_SIZE) return; /* no work needed */ - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; - - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; - - if (olditemsarr != NULL) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. - */ - for (i = 0; i < oldcap; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) + { + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); + } + + /* And release old hash table. */ + pfree(oldhash); } + } - /* And release old array. */ - pfree(olditemsarr); + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - Assert(resarr->nitems < resarr->maxitems); + Assert(owner->nhash < owner->grow_at); } /* - * Add a resource to ResourceArray + * Remember that an object is owner by a ReourceOwner * - * Caller must have previously done ResourceArrayEnlarge() + * Caller must have previously done ResourceOwnerEnlarge() */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 idx; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Append to linear array. */ - idx = resarr->nitems; - } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Remove a resource from ResourceArray + * Forget that an object is owned by a ResourceOwner * - * Returns true on success, false if resource was not found. + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. * - * Note: if same resource ID appears more than once, one instance is removed. + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 i, - idx, - lastidx = resarr->lastidx; + idx; - Assert(value != resarr->invalidval); +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) - { - if (resarr->itemsarr[i] == value) - { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + +#ifdef RESOWNER_STATS + narray_lookups++; +#endif + + return; } } - else + + /* Search hash */ + if (owner->nhash > 0) { - uint32 mask = resarr->capacity - 1; + uint32 mask = owner->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) + for (i = 0; i < owner->capacity; i++) { - if (resarr->itemsarr[idx] == value) + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; } idx = (idx + 1) & mask; } } - return false; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Call the ReleaseResource callback on entries with given 'phase'. */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +static void +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - if (resarr->nitems == 0) - return false; + bool found; + int capacity; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else + /* First handle all the entries in the array. */ + do { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; - - for (;;) + found = false; + for (int i = 0; i < owner->narr; i++) { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } } - } - *value = resarr->itemsarr[resarr->lastidx]; - return true; -} + /* + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. + */ + } while (found && owner->narr > 0); -/* - * Trash a ResourceArray (we don't care about its state after this) - */ -static void -ResourceArrayFree(ResourceArray *resarr) -{ - if (resarr->itemsarr) - pfree(resarr->itemsarr); + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) + { + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } + } + + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } @@ -437,17 +441,10 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name) parent->firstchild = owner; } - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif return owner; } @@ -486,6 +483,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +503,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +518,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +533,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +577,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +600,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - ReleaseCachedPlan(res, true); + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) + { + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + CurrentResourceOwner = save; } @@ -730,19 +652,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +676,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +830,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +881,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..1ed1580c644 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..281f85586fa 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,21 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +static inline void +ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} +static inline void +ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------3CC8131494C3F8E34BADAF02-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v4 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1212 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 803 insertions(+), 1052 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..34896b303ac 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all objects the same, but to register a +new kind of an object with it, you need to fill in a few callback +functions, see ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..99240679e09 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ - + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, true); + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +841,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +892,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------7349B9D66ED595EF3B1030B4-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v3 1/1] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 43 +- src/backend/storage/buffer/localbuf.c | 2 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 98 ++- src/backend/utils/cache/plancache.c | 50 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/cache/syscache.c | 2 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1139 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 15 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 21 files changed, 795 insertions(+), 1020 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c46b8abad12..8b77fb68941 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -739,7 +751,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); isExtend = (blockNum == P_NEW); @@ -1858,7 +1870,7 @@ BufferSync(int flags) WritebackContext wb_context; /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Unless this is a shutdown checkpoint or we have been explicitly told, @@ -2335,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context) */ /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); num_to_scan = bufs_to_lap; num_written = 0; @@ -3488,7 +3500,7 @@ FlushRelationBuffers(Relation rel) } /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3561,7 +3573,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3640,7 +3652,7 @@ FlushDatabaseBuffers(Oid dbid) BufferDesc *bufHdr; /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3738,7 +3750,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4802,3 +4814,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 04b3558ea33..811e0e3a45e 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index 931ed679307..5d2394e6780 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -96,7 +96,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -339,6 +339,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1429,7 +1447,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1611,7 +1629,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1741,7 +1759,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1779,7 +1797,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3635,3 +3653,19 @@ data_sync_elevel(int elevel) { return data_sync_retry ? elevel : PANIC; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index fa2b49c676e..24751c0d3f4 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,66 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* support for catcache refcount management */ +static inline void +ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs); +} +static inline void +ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs); +} + +static inline void +ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs); +} + +static inline void +ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs); +} + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1331,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1432,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1644,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1607,7 +1668,7 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ctlist = NIL; @@ -2062,14 +2123,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) { + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) +{ + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2083,9 +2149,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) +{ + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) { + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..484f14f5524 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +static inline void +ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} +static inline void +ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1254,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1417,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1476,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2230,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index e4dc4ee34ee..d6fe6a5bd5a 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -994,7 +994,7 @@ static const struct cachedesc cacheinfo[] = { } }; -static CatCache *SysCache[SysCacheSize]; + CatCache *SysCache[SysCacheSize]; static bool CacheInitialized = false; diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..890db7d1e66 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context references. +ResourceOwner treates all objects the same, but to register a new kind of +an object with it, you need to fill in a few callback functions, see +ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callabacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..c49758cb20d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) - -/* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. + * ResourceElem represents a reference associated with a resource owner. * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. - * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,45 +153,34 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ - -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* @@ -207,205 +189,227 @@ ResourceArrayInit(ResourceArray *resarr, Datum invalidval) * This is separate from actually inserting a resource because if we run out * of memory, it's critical to do so *before* acquiring the resource. */ -static void -ResourceArrayEnlarge(ResourceArray *resarr) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) + if (owner->narr < RESOWNER_ARRAY_SIZE) return; /* no work needed */ - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; - - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; - - if (olditemsarr != NULL) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. - */ - for (i = 0; i < oldcap; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) + { + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); + } + + /* And release old hash table. */ + pfree(oldhash); } + } - /* And release old array. */ - pfree(olditemsarr); + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - Assert(resarr->nitems < resarr->maxitems); + Assert(owner->nhash < owner->grow_at); } /* - * Add a resource to ResourceArray + * Remember that an object is owner by a ReourceOwner * - * Caller must have previously done ResourceArrayEnlarge() + * Caller must have previously done ResourceOwnerEnlarge() */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 idx; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Append to linear array. */ - idx = resarr->nitems; - } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Remove a resource from ResourceArray + * Forget that an object is owned by a ResourceOwner * - * Returns true on success, false if resource was not found. + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. * - * Note: if same resource ID appears more than once, one instance is removed. + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 i, - idx, - lastidx = resarr->lastidx; + idx; - Assert(value != resarr->invalidval); +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) - { - if (resarr->itemsarr[i] == value) - { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + +#ifdef RESOWNER_STATS + narray_lookups++; +#endif + + return; } } - else + + /* Search hash */ + if (owner->nhash > 0) { - uint32 mask = resarr->capacity - 1; + uint32 mask = owner->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) + for (i = 0; i < owner->capacity; i++) { - if (resarr->itemsarr[idx] == value) + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; } idx = (idx + 1) & mask; } } - return false; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Call the ReleaseResource callback on entries with given 'phase'. */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +static void +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - if (resarr->nitems == 0) - return false; + bool found; + int capacity; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else + /* First handle all the entries in the array. */ + do { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; - - for (;;) + found = false; + for (int i = 0; i < owner->narr; i++) { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } } - } - *value = resarr->itemsarr[resarr->lastidx]; - return true; -} + /* + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. + */ + } while (found && owner->narr > 0); -/* - * Trash a ResourceArray (we don't care about its state after this) - */ -static void -ResourceArrayFree(ResourceArray *resarr) -{ - if (resarr->itemsarr) - pfree(resarr->itemsarr); + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) + { + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } + } + + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } @@ -437,17 +441,10 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name) parent->firstchild = owner; } - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif return owner; } @@ -486,6 +483,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +503,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +518,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +533,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +577,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +600,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - ReleaseCachedPlan(res, true); + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) + { + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + CurrentResourceOwner = save; } @@ -730,19 +652,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +676,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +830,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +881,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..1ed1580c644 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..281f85586fa 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,21 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +static inline void +ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} +static inline void +ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------3CC8131494C3F8E34BADAF02-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v4 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1212 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 803 insertions(+), 1052 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..34896b303ac 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all objects the same, but to register a +new kind of an object with it, you need to fill in a few callback +functions, see ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..99240679e09 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ - + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, true); + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +841,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +892,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------7349B9D66ED595EF3B1030B4-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v5 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1221 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 811 insertions(+), 1057 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..c67ca603526 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resource the same, and extensions +can define a new kind of resource by defining a new ResourceOwnerFuncs +instance with a few callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..75fda4fcc13 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. + * Remember that an object is owner by a ReourceOwner * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; - - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +746,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +840,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +891,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------1104342304EC6341F5D7C60C Content-Type: text/x-patch; charset=UTF-8; name="0001-resownerbench.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-resownerbench.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v3 1/1] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 43 +- src/backend/storage/buffer/localbuf.c | 2 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 98 ++- src/backend/utils/cache/plancache.c | 50 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/cache/syscache.c | 2 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1139 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 15 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 21 files changed, 795 insertions(+), 1020 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c46b8abad12..8b77fb68941 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -739,7 +751,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); isExtend = (blockNum == P_NEW); @@ -1858,7 +1870,7 @@ BufferSync(int flags) WritebackContext wb_context; /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Unless this is a shutdown checkpoint or we have been explicitly told, @@ -2335,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context) */ /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); num_to_scan = bufs_to_lap; num_written = 0; @@ -3488,7 +3500,7 @@ FlushRelationBuffers(Relation rel) } /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3561,7 +3573,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3640,7 +3652,7 @@ FlushDatabaseBuffers(Oid dbid) BufferDesc *bufHdr; /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3738,7 +3750,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4802,3 +4814,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 04b3558ea33..811e0e3a45e 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index 931ed679307..5d2394e6780 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -96,7 +96,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -339,6 +339,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1429,7 +1447,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1611,7 +1629,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1741,7 +1759,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1779,7 +1797,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3635,3 +3653,19 @@ data_sync_elevel(int elevel) { return data_sync_retry ? elevel : PANIC; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index fa2b49c676e..24751c0d3f4 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,66 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* support for catcache refcount management */ +static inline void +ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs); +} +static inline void +ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs); +} + +static inline void +ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs); +} + +static inline void +ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs); +} + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1331,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1432,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1644,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1607,7 +1668,7 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ctlist = NIL; @@ -2062,14 +2123,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) { + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) +{ + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2083,9 +2149,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) +{ + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) { + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..484f14f5524 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +static inline void +ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} +static inline void +ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1254,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1417,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1476,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2230,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index e4dc4ee34ee..d6fe6a5bd5a 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -994,7 +994,7 @@ static const struct cachedesc cacheinfo[] = { } }; -static CatCache *SysCache[SysCacheSize]; + CatCache *SysCache[SysCacheSize]; static bool CacheInitialized = false; diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..890db7d1e66 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context references. +ResourceOwner treates all objects the same, but to register a new kind of +an object with it, you need to fill in a few callback functions, see +ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callabacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..c49758cb20d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) - -/* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. + * ResourceElem represents a reference associated with a resource owner. * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. - * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,45 +153,34 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ - -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* @@ -207,205 +189,227 @@ ResourceArrayInit(ResourceArray *resarr, Datum invalidval) * This is separate from actually inserting a resource because if we run out * of memory, it's critical to do so *before* acquiring the resource. */ -static void -ResourceArrayEnlarge(ResourceArray *resarr) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) + if (owner->narr < RESOWNER_ARRAY_SIZE) return; /* no work needed */ - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; - - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; - - if (olditemsarr != NULL) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. - */ - for (i = 0; i < oldcap; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) + { + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); + } + + /* And release old hash table. */ + pfree(oldhash); } + } - /* And release old array. */ - pfree(olditemsarr); + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - Assert(resarr->nitems < resarr->maxitems); + Assert(owner->nhash < owner->grow_at); } /* - * Add a resource to ResourceArray + * Remember that an object is owner by a ReourceOwner * - * Caller must have previously done ResourceArrayEnlarge() + * Caller must have previously done ResourceOwnerEnlarge() */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 idx; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Append to linear array. */ - idx = resarr->nitems; - } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Remove a resource from ResourceArray + * Forget that an object is owned by a ResourceOwner * - * Returns true on success, false if resource was not found. + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. * - * Note: if same resource ID appears more than once, one instance is removed. + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 i, - idx, - lastidx = resarr->lastidx; + idx; - Assert(value != resarr->invalidval); +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) - { - if (resarr->itemsarr[i] == value) - { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + +#ifdef RESOWNER_STATS + narray_lookups++; +#endif + + return; } } - else + + /* Search hash */ + if (owner->nhash > 0) { - uint32 mask = resarr->capacity - 1; + uint32 mask = owner->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) + for (i = 0; i < owner->capacity; i++) { - if (resarr->itemsarr[idx] == value) + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; } idx = (idx + 1) & mask; } } - return false; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Call the ReleaseResource callback on entries with given 'phase'. */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +static void +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - if (resarr->nitems == 0) - return false; + bool found; + int capacity; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else + /* First handle all the entries in the array. */ + do { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; - - for (;;) + found = false; + for (int i = 0; i < owner->narr; i++) { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } } - } - *value = resarr->itemsarr[resarr->lastidx]; - return true; -} + /* + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. + */ + } while (found && owner->narr > 0); -/* - * Trash a ResourceArray (we don't care about its state after this) - */ -static void -ResourceArrayFree(ResourceArray *resarr) -{ - if (resarr->itemsarr) - pfree(resarr->itemsarr); + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) + { + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } + } + + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } @@ -437,17 +441,10 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name) parent->firstchild = owner; } - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif return owner; } @@ -486,6 +483,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +503,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +518,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +533,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +577,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +600,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - ReleaseCachedPlan(res, true); + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) + { + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + CurrentResourceOwner = save; } @@ -730,19 +652,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +676,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +830,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +881,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..1ed1580c644 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..281f85586fa 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,21 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +static inline void +ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} +static inline void +ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------3CC8131494C3F8E34BADAF02-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v4 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1212 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 803 insertions(+), 1052 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..34896b303ac 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all objects the same, but to register a +new kind of an object with it, you need to fill in a few callback +functions, see ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..99240679e09 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ - + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, true); + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +841,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +892,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------7349B9D66ED595EF3B1030B4-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v3 1/1] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 43 +- src/backend/storage/buffer/localbuf.c | 2 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 98 ++- src/backend/utils/cache/plancache.c | 50 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/cache/syscache.c | 2 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1139 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 15 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 21 files changed, 795 insertions(+), 1020 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c46b8abad12..8b77fb68941 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -739,7 +751,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); isExtend = (blockNum == P_NEW); @@ -1858,7 +1870,7 @@ BufferSync(int flags) WritebackContext wb_context; /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Unless this is a shutdown checkpoint or we have been explicitly told, @@ -2335,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context) */ /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); num_to_scan = bufs_to_lap; num_written = 0; @@ -3488,7 +3500,7 @@ FlushRelationBuffers(Relation rel) } /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3561,7 +3573,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3640,7 +3652,7 @@ FlushDatabaseBuffers(Oid dbid) BufferDesc *bufHdr; /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3738,7 +3750,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4802,3 +4814,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 04b3558ea33..811e0e3a45e 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index 931ed679307..5d2394e6780 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -96,7 +96,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -339,6 +339,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1429,7 +1447,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1611,7 +1629,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1741,7 +1759,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1779,7 +1797,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3635,3 +3653,19 @@ data_sync_elevel(int elevel) { return data_sync_retry ? elevel : PANIC; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index fa2b49c676e..24751c0d3f4 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,66 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* support for catcache refcount management */ +static inline void +ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs); +} +static inline void +ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs); +} + +static inline void +ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs); +} + +static inline void +ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs); +} + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1331,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1432,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1644,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1607,7 +1668,7 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ctlist = NIL; @@ -2062,14 +2123,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) { + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) +{ + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2083,9 +2149,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) +{ + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) { + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..484f14f5524 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +static inline void +ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} +static inline void +ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1254,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1417,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1476,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2230,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index e4dc4ee34ee..d6fe6a5bd5a 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -994,7 +994,7 @@ static const struct cachedesc cacheinfo[] = { } }; -static CatCache *SysCache[SysCacheSize]; + CatCache *SysCache[SysCacheSize]; static bool CacheInitialized = false; diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..890db7d1e66 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context references. +ResourceOwner treates all objects the same, but to register a new kind of +an object with it, you need to fill in a few callback functions, see +ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callabacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..c49758cb20d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) - -/* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. + * ResourceElem represents a reference associated with a resource owner. * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. - * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,45 +153,34 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ - -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* @@ -207,205 +189,227 @@ ResourceArrayInit(ResourceArray *resarr, Datum invalidval) * This is separate from actually inserting a resource because if we run out * of memory, it's critical to do so *before* acquiring the resource. */ -static void -ResourceArrayEnlarge(ResourceArray *resarr) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) + if (owner->narr < RESOWNER_ARRAY_SIZE) return; /* no work needed */ - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; - - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; - - if (olditemsarr != NULL) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. - */ - for (i = 0; i < oldcap; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) + { + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); + } + + /* And release old hash table. */ + pfree(oldhash); } + } - /* And release old array. */ - pfree(olditemsarr); + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - Assert(resarr->nitems < resarr->maxitems); + Assert(owner->nhash < owner->grow_at); } /* - * Add a resource to ResourceArray + * Remember that an object is owner by a ReourceOwner * - * Caller must have previously done ResourceArrayEnlarge() + * Caller must have previously done ResourceOwnerEnlarge() */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 idx; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Append to linear array. */ - idx = resarr->nitems; - } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Remove a resource from ResourceArray + * Forget that an object is owned by a ResourceOwner * - * Returns true on success, false if resource was not found. + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. * - * Note: if same resource ID appears more than once, one instance is removed. + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 i, - idx, - lastidx = resarr->lastidx; + idx; - Assert(value != resarr->invalidval); +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) - { - if (resarr->itemsarr[i] == value) - { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + +#ifdef RESOWNER_STATS + narray_lookups++; +#endif + + return; } } - else + + /* Search hash */ + if (owner->nhash > 0) { - uint32 mask = resarr->capacity - 1; + uint32 mask = owner->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) + for (i = 0; i < owner->capacity; i++) { - if (resarr->itemsarr[idx] == value) + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; } idx = (idx + 1) & mask; } } - return false; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Call the ReleaseResource callback on entries with given 'phase'. */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +static void +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - if (resarr->nitems == 0) - return false; + bool found; + int capacity; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else + /* First handle all the entries in the array. */ + do { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; - - for (;;) + found = false; + for (int i = 0; i < owner->narr; i++) { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } } - } - *value = resarr->itemsarr[resarr->lastidx]; - return true; -} + /* + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. + */ + } while (found && owner->narr > 0); -/* - * Trash a ResourceArray (we don't care about its state after this) - */ -static void -ResourceArrayFree(ResourceArray *resarr) -{ - if (resarr->itemsarr) - pfree(resarr->itemsarr); + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) + { + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } + } + + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } @@ -437,17 +441,10 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name) parent->firstchild = owner; } - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif return owner; } @@ -486,6 +483,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +503,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +518,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +533,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +577,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +600,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - ReleaseCachedPlan(res, true); + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) + { + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + CurrentResourceOwner = save; } @@ -730,19 +652,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +676,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +830,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +881,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..1ed1580c644 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..281f85586fa 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,21 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +static inline void +ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} +static inline void +ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------3CC8131494C3F8E34BADAF02-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v4 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1212 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 803 insertions(+), 1052 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..34896b303ac 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all objects the same, but to register a +new kind of an object with it, you need to fill in a few callback +functions, see ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..99240679e09 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ - + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, true); + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +841,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +892,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------7349B9D66ED595EF3B1030B4-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v5 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1221 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 811 insertions(+), 1057 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..c67ca603526 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resource the same, and extensions +can define a new kind of resource by defining a new ResourceOwnerFuncs +instance with a few callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..75fda4fcc13 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. + * Remember that an object is owner by a ReourceOwner * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; - - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +746,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +840,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +891,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------1104342304EC6341F5D7C60C Content-Type: text/x-patch; charset=UTF-8; name="0001-resownerbench.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-resownerbench.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v5 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1221 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 811 insertions(+), 1057 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..c67ca603526 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resource the same, and extensions +can define a new kind of resource by defining a new ResourceOwnerFuncs +instance with a few callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..75fda4fcc13 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. + * Remember that an object is owner by a ReourceOwner * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; - - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +746,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +840,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +891,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------1104342304EC6341F5D7C60C Content-Type: text/x-patch; charset=UTF-8; name="0001-resownerbench.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-resownerbench.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v5 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1221 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 811 insertions(+), 1057 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..c67ca603526 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resource the same, and extensions +can define a new kind of resource by defining a new ResourceOwnerFuncs +instance with a few callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..75fda4fcc13 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. + * Remember that an object is owner by a ReourceOwner * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; - - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +746,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +840,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +891,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------1104342304EC6341F5D7C60C Content-Type: text/x-patch; charset=UTF-8; name="0001-resownerbench.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-resownerbench.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v4 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1212 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 803 insertions(+), 1052 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..34896b303ac 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all objects the same, but to register a +new kind of an object with it, you need to fill in a few callback +functions, see ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..99240679e09 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ - + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, true); + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +841,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +892,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------7349B9D66ED595EF3B1030B4-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v3 1/1] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 43 +- src/backend/storage/buffer/localbuf.c | 2 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 98 ++- src/backend/utils/cache/plancache.c | 50 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/cache/syscache.c | 2 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1139 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 15 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 41 +- src/include/utils/resowner_private.h | 105 --- 21 files changed, 795 insertions(+), 1020 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c46b8abad12..8b77fb68941 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -739,7 +751,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); isExtend = (blockNum == P_NEW); @@ -1858,7 +1870,7 @@ BufferSync(int flags) WritebackContext wb_context; /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Unless this is a shutdown checkpoint or we have been explicitly told, @@ -2335,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context) */ /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); num_to_scan = bufs_to_lap; num_written = 0; @@ -3488,7 +3500,7 @@ FlushRelationBuffers(Relation rel) } /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3561,7 +3573,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3640,7 +3652,7 @@ FlushDatabaseBuffers(Oid dbid) BufferDesc *bufHdr; /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); for (i = 0; i < NBuffers; i++) { @@ -3738,7 +3750,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4802,3 +4814,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 04b3558ea33..811e0e3a45e 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index 931ed679307..5d2394e6780 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -96,7 +96,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -339,6 +339,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1429,7 +1447,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1611,7 +1629,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1741,7 +1759,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1779,7 +1797,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3635,3 +3653,19 @@ data_sync_elevel(int elevel) { return data_sync_retry ? elevel : PANIC; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index fa2b49c676e..24751c0d3f4 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,66 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* support for catcache refcount management */ +static inline void +ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs); +} +static inline void +ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) +{ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs); +} + +static inline void +ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) +{ + ResourceOwnerEnlarge(owner); +} +static inline void +ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs); +} + +static inline void +ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) +{ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs); +} + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1331,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1432,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1644,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1607,7 +1668,7 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ctlist = NIL; @@ -2062,14 +2123,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) { + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) +{ + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2083,9 +2149,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) +{ + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) { + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..484f14f5524 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +static inline void +ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} +static inline void +ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) +{ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs); +} + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1254,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1417,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1476,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2230,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index e4dc4ee34ee..d6fe6a5bd5a 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -994,7 +994,7 @@ static const struct cachedesc cacheinfo[] = { } }; -static CatCache *SysCache[SysCacheSize]; + CatCache *SysCache[SysCacheSize]; static bool CacheInitialized = false; diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..890db7d1e66 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context references. +ResourceOwner treates all objects the same, but to register a new kind of +an object with it, you need to fill in a few callback functions, see +ResourceOwnerFuncs. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callabacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..c49758cb20d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) - -/* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. + * ResourceElem represents a reference associated with a resource owner. * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. - * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,45 +153,34 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ - -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* @@ -207,205 +189,227 @@ ResourceArrayInit(ResourceArray *resarr, Datum invalidval) * This is separate from actually inserting a resource because if we run out * of memory, it's critical to do so *before* acquiring the resource. */ -static void -ResourceArrayEnlarge(ResourceArray *resarr) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) + if (owner->narr < RESOWNER_ARRAY_SIZE) return; /* no work needed */ - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; - - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; - - if (olditemsarr != NULL) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. - */ - for (i = 0; i < oldcap; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) + { + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); + } + + /* And release old hash table. */ + pfree(oldhash); } + } - /* And release old array. */ - pfree(olditemsarr); + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - Assert(resarr->nitems < resarr->maxitems); + Assert(owner->nhash < owner->grow_at); } /* - * Add a resource to ResourceArray + * Remember that an object is owner by a ReourceOwner * - * Caller must have previously done ResourceArrayEnlarge() + * Caller must have previously done ResourceOwnerEnlarge() */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 idx; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Append to linear array. */ - idx = resarr->nitems; - } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Remove a resource from ResourceArray + * Forget that an object is owned by a ResourceOwner * - * Returns true on success, false if resource was not found. + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. * - * Note: if same resource ID appears more than once, one instance is removed. + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { uint32 i, - idx, - lastidx = resarr->lastidx; + idx; - Assert(value != resarr->invalidval); +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) - { - if (resarr->itemsarr[i] == value) - { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + +#ifdef RESOWNER_STATS + narray_lookups++; +#endif + + return; } } - else + + /* Search hash */ + if (owner->nhash > 0) { - uint32 mask = resarr->capacity - 1; + uint32 mask = owner->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) + for (i = 0; i < owner->capacity; i++) { - if (resarr->itemsarr[idx] == value) + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; } idx = (idx + 1) & mask; } } - return false; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Call the ReleaseResource callback on entries with given 'phase'. */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +static void +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - if (resarr->nitems == 0) - return false; + bool found; + int capacity; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else + /* First handle all the entries in the array. */ + do { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; - - for (;;) + found = false; + for (int i = 0; i < owner->narr; i++) { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } } - } - *value = resarr->itemsarr[resarr->lastidx]; - return true; -} + /* + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. + */ + } while (found && owner->narr > 0); -/* - * Trash a ResourceArray (we don't care about its state after this) - */ -static void -ResourceArrayFree(ResourceArray *resarr) -{ - if (resarr->itemsarr) - pfree(resarr->itemsarr); + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) + { + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } + } + + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } @@ -437,17 +441,10 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name) parent->firstchild = owner; } - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif return owner; } @@ -486,6 +483,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +503,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +518,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +533,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +577,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +600,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - ReleaseCachedPlan(res, true); + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) + { + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + CurrentResourceOwner = save; } @@ -730,19 +652,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +676,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -922,44 +830,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +881,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..1ed1580c644 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..281f85586fa 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,21 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +static inline void +ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} +static inline void +ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) +{ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs); +} + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..c3f1f060fa9 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,32 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an object of a specific kind are encapsulated in + * ResourceOwnerFuncs. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +97,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------3CC8131494C3F8E34BADAF02-- ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v5 2/2] Make resowners more easily extensible. @ 2020-12-16 15:26 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2020-12-16 15:26 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 40 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 39 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1221 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 45 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- 20 files changed, 811 insertions(+), 1057 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..7fa44d5f7ee 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,21 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; /* * CreateTemplateTupleDesc @@ -376,9 +388,10 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; - ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); } /* @@ -394,7 +407,8 @@ DecrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount > 0); - ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(tupdesc), + &tupdesc_resowner_funcs); if (--tupdesc->tdrefcount == 0) FreeTupleDesc(tupdesc); } @@ -925,3 +939,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..64c7fd92bc7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,20 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +163,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +171,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(context), &jit_funcs); return context; } @@ -221,6 +233,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForget(context->resowner, PointerGetDatum(context), &jit_funcs); } /* @@ -1231,3 +1245,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..185ff00ee76 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..255d19a88b9 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,18 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,10 +2102,10 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -2105,7 +2118,7 @@ RelationDecrementReferenceCount(Relation rel) Assert(rel->rd_refcnt > 0); rel->rd_refcnt -= 1; if (!IsBootstrapProcessingMode()) - ResourceOwnerForgetRelationRef(CurrentResourceOwner, rel); + ResourceOwnerForget(CurrentResourceOwner, PointerGetDatum(rel), &relref_resowner_funcs); } /* @@ -6417,3 +6430,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..c67ca603526 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of objects. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resource the same, and extensions +can define a new kind of resource by defining a new ResourceOwnerFuncs +instance with a few callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..75fda4fcc13 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -22,73 +22,44 @@ #include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +89,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple + * array. When the array gets full, all the elements in the array are + * moved to a hash table. This way, the array always contains a few + * most recently remembered references. To find a particular reference, + * you need to search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +127,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +153,311 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceArrayAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop + * to check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved + * around. If that happened, scan the hash table again, so that we + * don't miss entries that were moved. (XXX: I'm not sure if any of + * the callbacks actually do that, but this is cheap to check, and + * better safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* We assume we can't fail below this point, so OK to scribble on FIXME */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they don't + * necessarily go where they were before, so this simple logic is the + * best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceArrayAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + for (int i = 0; i < owner->narr; i++) + { + ResourceArrayAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. - * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. + * Remember that an object is owner by a ReourceOwner * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} + uint32 i, + idx; +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; - - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource owner + * are pointers. It's a bit misleading if it's not a pointer, but this is a + * programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +494,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +514,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +529,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that + * During a commit, there shouldn't be any remaining references --- that * would indicate failure to clean up the executor correctly --- so * issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +544,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +588,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +611,42 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* pass 'false' because we already removed the entry from the resowner */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +663,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +687,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +746,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +840,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +891,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..10cec8d39d0 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,18 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + /* * Snapshot fields to be serialized. * @@ -831,9 +843,10 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; - ResourceOwnerRememberSnapshot(owner, snap); + ResourceOwnerRemember(owner, PointerGetDatum(snap), + &snapshot_resowner_funcs); if (snap->regd_count == 1) pairingheap_add(&RegisteredSnapshots, &snap->ph_node); @@ -870,7 +883,8 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner) Assert(snapshot->regd_count > 0); Assert(!pairingheap_is_empty(&RegisteredSnapshots)); - ResourceOwnerForgetSnapshot(owner, snapshot); + ResourceOwnerForget(owner, PointerGetDatum(snapshot), + &snapshot_resowner_funcs); snapshot->regd_count--; if (snapshot->regd_count == 0) @@ -2345,3 +2359,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..642e72d8c0f 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,21 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold JitContexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; +#endif + /* * pg_cryptohash_create * @@ -77,7 +91,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +120,8 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRemember(CurrentResourceOwner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif return ctx; @@ -207,10 +221,29 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForget(ctx->resowner, PointerGetDatum(ctx), + &cryptohash_funcs); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ -- 2.29.2 --------------1104342304EC6341F5D7C60C Content-Type: text/x-patch; charset=UTF-8; name="0001-resownerbench.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-resownerbench.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v6 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1236 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 844 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..8e5b86a0e5b 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..9f9db426274 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..1db1e7cf14b 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +614,48 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +672,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +696,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +755,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +849,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +900,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..3ded16f6fc8 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2345,3 +2363,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..6421f1de5cd 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -77,7 +97,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +126,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -207,10 +226,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 943142ced8c..2df8d3cbefa 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2127,8 +2127,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.29.2 --------------F30A5443A5D524A5CBC51AE7 Content-Type: text/x-patch; charset=UTF-8; name="v6-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v6-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v8 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 47 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1312 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/common/hmac_openssl.c | 47 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 112 --- src/tools/pgindent/typedefs.list | 4 +- 22 files changed, 894 insertions(+), 1137 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 4c63bd4dc64..f2d4442c938 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -30,9 +30,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -367,7 +385,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -910,3 +928,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index df691cbf1c5..57e3144fae7 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1234,3 +1254,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index 2383e4cac08..9e55ad3f743 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -625,7 +637,7 @@ ReadRecentBuffer(RelFileNode rnode, ForkNumber forkNum, BlockNumber blockNum, Assert(BufferIsValid(recent_buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ReservePrivateRefCountEntry(); INIT_BUFFERTAG(tag, rnode, forkNum, blockNum); @@ -1166,7 +1178,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1677,7 +1689,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1760,7 +1772,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2501,7 +2513,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3568,7 +3580,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3663,7 +3675,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3718,7 +3730,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3801,7 +3813,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4848,3 +4860,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index a340a5f6afe..6a9ee97ea1e 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -99,7 +99,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -345,6 +345,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1449,7 +1467,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1631,7 +1649,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1761,7 +1779,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1799,7 +1817,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3779,3 +3797,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 13eed587601..ed046db57ec 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1272,7 +1309,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1373,7 +1410,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1585,7 +1622,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1697,7 +1734,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2071,14 +2108,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2092,9 +2134,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 6767eae8f20..a10362629f8 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 13d9994af3e..90d2892489f 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -77,13 +77,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2056,6 +2057,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2067,7 +2086,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6546,3 +6565,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index e24f00f0601..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,76 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "common/hmac.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -119,24 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ - ResourceArray hmacarr; /* HMAC contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -148,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -162,298 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); -static void PrintHMACLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->hmacarr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -490,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -501,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -517,71 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } - - /* Ditto for HMAC contexts */ - while (ResourceArrayGetAny(&(owner->hmacarr), &foundres)) - { - pg_hmac_ctx *context = (pg_hmac_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintHMACLeakWarning(foundres); - pg_hmac_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -590,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -634,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -717,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -740,20 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); - Assert(owner->hmacarr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -769,19 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - ResourceArrayFree(&(owner->hmacarr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -839,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -934,58 +842,20 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - /* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. + * Remember that a Local Lock is owned by a ResourceOwner * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * This is different from the other Remember functions in that the list of + * locks is only a lossy cache. It can hold up to MAX_RESOWNER_LOCKS entries, + * and when it overflows, we stop tracking locks. The point of only remembering + * only up to MAX_RESOWNER_LOCKS entries is that if a lot of locks are held, + * ResourceOwnerForgetLock doesn't need to scan through a large array to find + * the entry. */ void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) +ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock) { - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - -/* - * Remember that a Local Lock is owned by a ResourceOwner - * - * This is different from the other Remember functions in that the list of - * locks is only a lossy cache. It can hold up to MAX_RESOWNER_LOCKS entries, - * and when it overflows, we stop tracking locks. The point of only remembering - * only up to MAX_RESOWNER_LOCKS entries is that if a lot of locks are held, - * ResourceOwnerForgetLock doesn't need to scan through a large array to find - * the entry. - */ -void -ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock) -{ - Assert(locallock != NULL); + Assert(locallock != NULL); if (owner->nlocks > MAX_RESOWNER_LOCKS) return; /* we have already overflowed */ @@ -1023,469 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * hmac context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeHMAC(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->hmacarr)); -} - -/* - * Remember that a HMAC context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeHMAC() - */ -void -ResourceOwnerRememberHMAC(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->hmacarr), handle); -} - -/* - * Forget that a HMAC context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetHMAC(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->hmacarr), handle)) - elog(ERROR, "HMAC context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintHMACLeakWarning(Datum handle) -{ - elog(WARNING, "HMAC context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2968c7f7b7d..1f46cbe58c9 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/common/hmac_openssl.c b/src/common/hmac_openssl.c index 1acf59476eb..0a65284bae9 100644 --- a/src/common/hmac_openssl.c +++ b/src/common/hmac_openssl.c @@ -29,7 +29,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_hmac_ctx #endif }; +/* ResourceOwner callbacks to hold HMAC contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseHMAC(Datum res); +static void ResOwnerPrintHMACLeakWarning(Datum res); + +static ResourceOwnerFuncs hmac_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL HMAC context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseHMAC, + .PrintLeakWarning = ResOwnerPrintHMACLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberHMAC(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &hmac_resowner_funcs) +#define ResourceOwnerForgetHMAC(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &hmac_resowner_funcs) +#endif + /* * pg_hmac_create * @@ -86,7 +106,7 @@ pg_hmac_create(pg_cryptohash_type type) */ #ifdef HAVE_HMAC_CTX_NEW #ifndef FRONTEND - ResourceOwnerEnlargeHMAC(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx->hmacctx = HMAC_CTX_new(); #else @@ -108,7 +128,7 @@ pg_hmac_create(pg_cryptohash_type type) #ifdef HAVE_HMAC_CTX_NEW #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberHMAC(CurrentResourceOwner, PointerGetDatum(ctx)); + ResourceOwnerRememberHMAC(CurrentResourceOwner, ctx); #endif #else memset(ctx->hmacctx, 0, sizeof(HMAC_CTX)); @@ -244,7 +264,7 @@ pg_hmac_free(pg_hmac_ctx *ctx) #ifdef HAVE_HMAC_CTX_FREE HMAC_CTX_free(ctx->hmacctx); #ifndef FRONTEND - ResourceOwnerForgetHMAC(ctx->resowner, PointerGetDatum(ctx)); + ResourceOwnerForgetHMAC(ctx->resowner, ctx); #endif #else explicit_bzero(ctx->hmacctx, sizeof(HMAC_CTX)); @@ -254,3 +274,22 @@ pg_hmac_free(pg_hmac_ctx *ctx) explicit_bzero(ctx, sizeof(pg_hmac_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseHMAC(Datum res) +{ + pg_hmac_free((pg_hmac_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintHMACLeakWarning(Datum res) +{ + elog(WARNING, "HMAC context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index 33fcaf5c9a8..10ef8d850e4 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -300,6 +300,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index 6dafc87e28c..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,112 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -/* support for HMAC context management */ -extern void ResourceOwnerEnlargeHMAC(ResourceOwner owner); -extern void ResourceOwnerRememberHMAC(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetHMAC(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 37cf4b2f76b..1a0e05df62b 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2223,8 +2223,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.2 --------------846F221A8A016B7AA8E567B3 Content-Type: text/x-patch; charset=UTF-8; name="v8-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v8-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v6 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1236 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 844 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..8e5b86a0e5b 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..9f9db426274 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..1db1e7cf14b 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +614,48 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +672,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +696,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +755,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +849,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +900,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..3ded16f6fc8 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2345,3 +2363,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..6421f1de5cd 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -77,7 +97,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +126,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -207,10 +226,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 943142ced8c..2df8d3cbefa 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2127,8 +2127,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.29.2 --------------F30A5443A5D524A5CBC51AE7 Content-Type: text/x-patch; charset=UTF-8; name="v6-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v6-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v6 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1236 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 844 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..8e5b86a0e5b 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..9f9db426274 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..1db1e7cf14b 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +614,48 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +672,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +696,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +755,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +849,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +900,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..3ded16f6fc8 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2345,3 +2363,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..6421f1de5cd 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -77,7 +97,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +126,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -207,10 +226,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 943142ced8c..2df8d3cbefa 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2127,8 +2127,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.29.2 --------------F30A5443A5D524A5CBC51AE7 Content-Type: text/x-patch; charset=UTF-8; name="v6-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v6-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v6 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1236 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 844 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..8e5b86a0e5b 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..9f9db426274 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..1db1e7cf14b 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +614,48 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +672,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +696,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +755,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +849,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +900,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..3ded16f6fc8 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2345,3 +2363,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..6421f1de5cd 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -77,7 +97,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +126,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -207,10 +226,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 943142ced8c..2df8d3cbefa 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2127,8 +2127,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.29.2 --------------F30A5443A5D524A5CBC51AE7 Content-Type: text/x-patch; charset=UTF-8; name="v6-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v6-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v6 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1236 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 844 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..8e5b86a0e5b 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..9f9db426274 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..1db1e7cf14b 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +614,48 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +672,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +696,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +755,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +849,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +900,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..3ded16f6fc8 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2345,3 +2363,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..6421f1de5cd 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -77,7 +97,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +126,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -207,10 +226,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 943142ced8c..2df8d3cbefa 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2127,8 +2127,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.29.2 --------------F30A5443A5D524A5CBC51AE7 Content-Type: text/x-patch; charset=UTF-8; name="v6-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v6-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v6 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1236 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 844 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..8e5b86a0e5b 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..9f9db426274 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..1db1e7cf14b 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +614,48 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +672,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +696,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +755,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +849,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +900,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..3ded16f6fc8 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2345,3 +2363,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..6421f1de5cd 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -77,7 +97,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +126,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -207,10 +226,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 943142ced8c..2df8d3cbefa 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2127,8 +2127,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.29.2 --------------F30A5443A5D524A5CBC51AE7 Content-Type: text/x-patch; charset=UTF-8; name="v6-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v6-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v6 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1236 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 844 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..8e5b86a0e5b 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..9f9db426274 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..1db1e7cf14b 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +614,48 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +672,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +696,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +755,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +849,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +900,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..3ded16f6fc8 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2345,3 +2363,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..6421f1de5cd 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -77,7 +97,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +126,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -207,10 +226,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 943142ced8c..2df8d3cbefa 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2127,8 +2127,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.29.2 --------------F30A5443A5D524A5CBC51AE7 Content-Type: text/x-patch; charset=UTF-8; name="v6-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v6-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v6 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1236 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 844 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..8e5b86a0e5b 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..9f9db426274 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..1db1e7cf14b 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +614,48 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +672,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +696,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +755,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +849,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +900,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..3ded16f6fc8 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2345,3 +2363,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..6421f1de5cd 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -77,7 +97,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +126,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -207,10 +226,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 943142ced8c..2df8d3cbefa 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2127,8 +2127,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.29.2 --------------F30A5443A5D524A5CBC51AE7 Content-Type: text/x-patch; charset=UTF-8; name="v6-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v6-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v6 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1236 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 844 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..8e5b86a0e5b 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..9f9db426274 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..1db1e7cf14b 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +614,48 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +672,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +696,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +755,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +849,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +900,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..3ded16f6fc8 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2345,3 +2363,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..6421f1de5cd 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -77,7 +97,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +126,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -207,10 +226,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 943142ced8c..2df8d3cbefa 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2127,8 +2127,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.29.2 --------------F30A5443A5D524A5CBC51AE7 Content-Type: text/x-patch; charset=UTF-8; name="v6-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v6-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v6 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1236 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 844 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..8e5b86a0e5b 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..9f9db426274 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..1db1e7cf14b 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +614,48 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +672,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +696,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +755,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +849,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +900,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..3ded16f6fc8 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2345,3 +2363,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..6421f1de5cd 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -77,7 +97,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +126,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -207,10 +226,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 943142ced8c..2df8d3cbefa 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2127,8 +2127,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.29.2 --------------F30A5443A5D524A5CBC51AE7 Content-Type: text/x-patch; charset=UTF-8; name="v6-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v6-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v6 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1236 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 844 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..8e5b86a0e5b 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..9f9db426274 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..1db1e7cf14b 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +614,48 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +672,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +696,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +755,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +849,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +900,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..3ded16f6fc8 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2345,3 +2363,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..6421f1de5cd 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -77,7 +97,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +126,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -207,10 +226,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 943142ced8c..2df8d3cbefa 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2127,8 +2127,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.29.2 --------------F30A5443A5D524A5CBC51AE7 Content-Type: text/x-patch; charset=UTF-8; name="v6-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v6-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v6 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1236 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 844 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..8e5b86a0e5b 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..9f9db426274 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..1db1e7cf14b 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +614,48 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +672,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +696,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +755,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +849,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +900,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..3ded16f6fc8 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2345,3 +2363,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..6421f1de5cd 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -77,7 +97,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +126,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -207,10 +226,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 943142ced8c..2df8d3cbefa 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2127,8 +2127,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.29.2 --------------F30A5443A5D524A5CBC51AE7 Content-Type: text/x-patch; charset=UTF-8; name="v6-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v6-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v6 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1236 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 844 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..8e5b86a0e5b 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..9f9db426274 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..1db1e7cf14b 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +614,48 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +672,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +696,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +755,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +849,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +900,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..3ded16f6fc8 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2345,3 +2363,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..6421f1de5cd 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -77,7 +97,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +126,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -207,10 +226,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 943142ced8c..2df8d3cbefa 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2127,8 +2127,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.29.2 --------------F30A5443A5D524A5CBC51AE7 Content-Type: text/x-patch; charset=UTF-8; name="v6-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v6-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v6 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1236 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 844 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..8e5b86a0e5b 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..9f9db426274 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..1db1e7cf14b 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +614,48 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +672,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +696,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +755,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +849,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +900,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..3ded16f6fc8 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2345,3 +2363,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..6421f1de5cd 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -77,7 +97,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +126,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -207,10 +226,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 943142ced8c..2df8d3cbefa 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2127,8 +2127,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.29.2 --------------F30A5443A5D524A5CBC51AE7 Content-Type: text/x-patch; charset=UTF-8; name="v6-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v6-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v6 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1236 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 844 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..8e5b86a0e5b 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..9f9db426274 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..1db1e7cf14b 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +614,48 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +672,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +696,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +755,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +849,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +900,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..3ded16f6fc8 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2345,3 +2363,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..6421f1de5cd 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -77,7 +97,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +126,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -207,10 +226,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 943142ced8c..2df8d3cbefa 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2127,8 +2127,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.29.2 --------------F30A5443A5D524A5CBC51AE7 Content-Type: text/x-patch; charset=UTF-8; name="v6-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v6-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v6 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1236 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 844 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..8e5b86a0e5b 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..9f9db426274 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..1db1e7cf14b 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +614,48 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +672,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +696,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +755,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +849,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +900,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..3ded16f6fc8 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2345,3 +2363,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..6421f1de5cd 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -77,7 +97,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +126,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -207,10 +226,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 943142ced8c..2df8d3cbefa 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2127,8 +2127,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.29.2 --------------F30A5443A5D524A5CBC51AE7 Content-Type: text/x-patch; charset=UTF-8; name="v6-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v6-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v6 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1236 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 844 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..8e5b86a0e5b 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..9f9db426274 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..1db1e7cf14b 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +614,48 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +672,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +696,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +755,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +849,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +900,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..3ded16f6fc8 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2345,3 +2363,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..6421f1de5cd 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -77,7 +97,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +126,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -207,10 +226,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 943142ced8c..2df8d3cbefa 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2127,8 +2127,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.29.2 --------------F30A5443A5D524A5CBC51AE7 Content-Type: text/x-patch; charset=UTF-8; name="v6-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v6-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v6 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1236 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 844 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..8e5b86a0e5b 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..9f9db426274 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..1db1e7cf14b 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +614,48 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +672,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +696,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +755,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +849,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +900,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..3ded16f6fc8 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2345,3 +2363,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..6421f1de5cd 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -77,7 +97,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +126,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -207,10 +226,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 943142ced8c..2df8d3cbefa 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2127,8 +2127,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.29.2 --------------F30A5443A5D524A5CBC51AE7 Content-Type: text/x-patch; charset=UTF-8; name="v6-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v6-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v6 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1236 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 844 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..8e5b86a0e5b 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..9f9db426274 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..1db1e7cf14b 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +614,48 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +672,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +696,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +755,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +849,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +900,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..3ded16f6fc8 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2345,3 +2363,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..6421f1de5cd 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -77,7 +97,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +126,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -207,10 +226,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 943142ced8c..2df8d3cbefa 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2127,8 +2127,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.29.2 --------------F30A5443A5D524A5CBC51AE7 Content-Type: text/x-patch; charset=UTF-8; name="v6-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v6-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v10 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 47 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 17 +- src/backend/utils/resowner/resowner.c | 1323 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/common/hmac_openssl.c | 47 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 112 --- src/tools/pgindent/typedefs.list | 4 +- 22 files changed, 907 insertions(+), 1133 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 4c63bd4dc64..f2d4442c938 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -30,9 +30,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -367,7 +385,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -910,3 +928,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 91b8ae6c51a..d42d1bc6c48 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 169dad96d76..c4c766d8937 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1248,3 +1268,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde9184c7d6..d563df548ff 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -625,7 +637,7 @@ ReadRecentBuffer(RelFileNode rnode, ForkNumber forkNum, BlockNumber blockNum, Assert(BufferIsValid(recent_buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ReservePrivateRefCountEntry(); INIT_BUFFERTAG(tag, rnode, forkNum, blockNum); @@ -1175,7 +1187,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1686,7 +1698,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1769,7 +1781,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2510,7 +2522,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3564,7 +3576,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3659,7 +3671,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3714,7 +3726,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3797,7 +3809,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4844,3 +4856,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index f9cda6906d2..be5576451d5 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -99,7 +99,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -350,6 +350,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1529,7 +1547,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1713,7 +1731,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1845,7 +1863,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) Assert(temporary_files_allowed); /* check temp file access is up */ - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1885,7 +1903,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) Assert(temporary_files_allowed); /* check temp file access is up */ - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3870,3 +3888,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 364654e1060..933f9121be5 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -48,7 +48,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 13eed587601..ed046db57ec 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1272,7 +1309,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1373,7 +1410,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1585,7 +1622,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1697,7 +1734,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2071,14 +2108,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2092,9 +2134,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 6767eae8f20..a10362629f8 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 13d9994af3e..90d2892489f 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -77,13 +77,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2056,6 +2057,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2067,7 +2086,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6546,3 +6565,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index f94c9700df4..adaa4a9d00e 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -54,12 +54,17 @@ The basic operations on a ResourceOwner are: * delete a ResourceOwner (including child owner objects); all resources must have been released beforehand -This API directly supports the resource types listed in the definition of -ResourceOwnerData struct in src/backend/utils/resowner/resowner.c. -Other objects can be associated with a ResourceOwner by recording the address -of the owning ResourceOwner in such an object. There is an API for other -modules to get control during ResourceOwner release, so that they can scan -their own data structures to find the objects that need to be deleted. +ResourceOwner can record ownership of many different kinds of resources. In +core PostgreSQL, it is used for buffer pins, lmgr locks, and catalog cache +references, to name a few examples. ResourceOwner treats all resources the +same, and extensions can define new kinds of resources by filling in a +ResourceOwnerFuncs struct with suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Locks are handled specially because in non-error situations a lock should be held until end of transaction, even if it was originally taken by a diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index e24f00f0601..35e824d4534 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,76 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "common/hmac.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; /* NULL indicates a free hash table slot */ +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -119,24 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ - ResourceArray hmacarr; /* HMAC contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -148,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -162,298 +152,332 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); +static inline uint32 hash_resource_elem(Datum value, ResourceOwnerFuncs *kind); +static void ResourceOwnerAddToHash(ResourceOwner owner, Datum value, + ResourceOwnerFuncs *kind); +static void ResourceOwnerReleaseAll(ResourceOwner owner, + ResourceReleasePhase phase, + bool printLeakWarnings); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); -static void PrintHMACLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} /* - * Initialize a ResourceArray + * Adds 'value' of given 'kind' to the ResourceOwner's hash table */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + Assert(kind != NULL); + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; /* found a free slot */ + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; + bool found; + int capacity; - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ - - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* + * First handle all the entries in the array. + * + * Note that ReleaseResource() will call ResourceOwnerForget) and remove + * the entry from our array, so we just have to iterate till there is + * nothing left to remove. + */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* + * Ok, the array has now been handled. Then the hash. Like with the + * array, ReleaseResource() will remove the entry from the hash. + */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the same resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the + * callbacks actually do that, but this is cheap to check, and better + * safe than sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->hmacarr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -490,6 +514,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -501,7 +534,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -517,71 +549,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all resources that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining resources --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } - - /* Ditto for HMAC contexts */ - while (ResourceArrayGetAny(&(owner->hmacarr), &foundres)) - { - pg_hmac_ctx *context = (pg_hmac_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintHMACLeakWarning(foundres); - pg_hmac_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -634,70 +608,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all resources that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -717,13 +630,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, owner); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -740,20 +682,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); - Assert(owner->hmacarr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -769,19 +706,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - ResourceArrayFree(&(owner->hmacarr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -839,11 +765,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -934,58 +859,20 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - /* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. + * Remember that a Local Lock is owned by a ResourceOwner * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * This is different from the other Remember functions in that the list of + * locks is only a lossy cache. It can hold up to MAX_RESOWNER_LOCKS entries, + * and when it overflows, we stop tracking locks. The point of only remembering + * only up to MAX_RESOWNER_LOCKS entries is that if a lot of locks are held, + * ResourceOwnerForgetLock doesn't need to scan through a large array to find + * the entry. */ void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) +ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock) { - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - -/* - * Remember that a Local Lock is owned by a ResourceOwner - * - * This is different from the other Remember functions in that the list of - * locks is only a lossy cache. It can hold up to MAX_RESOWNER_LOCKS entries, - * and when it overflows, we stop tracking locks. The point of only remembering - * only up to MAX_RESOWNER_LOCKS entries is that if a lot of locks are held, - * ResourceOwnerForgetLock doesn't need to scan through a large array to find - * the entry. - */ -void -ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock) -{ - Assert(locallock != NULL); + Assert(locallock != NULL); if (owner->nlocks > MAX_RESOWNER_LOCKS) return; /* we have already overflowed */ @@ -1023,469 +910,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * hmac context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeHMAC(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->hmacarr)); -} - -/* - * Remember that a HMAC context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeHMAC() - */ -void -ResourceOwnerRememberHMAC(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->hmacarr), handle); -} - -/* - * Forget that a HMAC context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetHMAC(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->hmacarr), handle)) - elog(ERROR, "HMAC context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintHMACLeakWarning(Datum handle) -{ - elog(WARNING, "HMAC context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 2968c7f7b7d..1f46cbe58c9 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/common/hmac_openssl.c b/src/common/hmac_openssl.c index 1acf59476eb..0a65284bae9 100644 --- a/src/common/hmac_openssl.c +++ b/src/common/hmac_openssl.c @@ -29,7 +29,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_hmac_ctx #endif }; +/* ResourceOwner callbacks to hold HMAC contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseHMAC(Datum res); +static void ResOwnerPrintHMACLeakWarning(Datum res); + +static ResourceOwnerFuncs hmac_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL HMAC context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseHMAC, + .PrintLeakWarning = ResOwnerPrintHMACLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberHMAC(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &hmac_resowner_funcs) +#define ResourceOwnerForgetHMAC(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &hmac_resowner_funcs) +#endif + /* * pg_hmac_create * @@ -86,7 +106,7 @@ pg_hmac_create(pg_cryptohash_type type) */ #ifdef HAVE_HMAC_CTX_NEW #ifndef FRONTEND - ResourceOwnerEnlargeHMAC(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx->hmacctx = HMAC_CTX_new(); #else @@ -108,7 +128,7 @@ pg_hmac_create(pg_cryptohash_type type) #ifdef HAVE_HMAC_CTX_NEW #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberHMAC(CurrentResourceOwner, PointerGetDatum(ctx)); + ResourceOwnerRememberHMAC(CurrentResourceOwner, ctx); #endif #else memset(ctx->hmacctx, 0, sizeof(HMAC_CTX)); @@ -244,7 +264,7 @@ pg_hmac_free(pg_hmac_ctx *ctx) #ifdef HAVE_HMAC_CTX_FREE HMAC_CTX_free(ctx->hmacctx); #ifndef FRONTEND - ResourceOwnerForgetHMAC(ctx->resowner, PointerGetDatum(ctx)); + ResourceOwnerForgetHMAC(ctx->resowner, ctx); #endif #else explicit_bzero(ctx->hmacctx, sizeof(HMAC_CTX)); @@ -254,3 +274,22 @@ pg_hmac_free(pg_hmac_ctx *ctx) explicit_bzero(ctx, sizeof(pg_hmac_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseHMAC(Datum res) +{ + pg_hmac_free((pg_hmac_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintHMACLeakWarning(Datum res) +{ + elog(WARNING, "HMAC context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index 33fcaf5c9a8..10ef8d850e4 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -300,6 +300,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index 6dafc87e28c..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,112 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -/* support for HMAC context management */ -extern void ResourceOwnerEnlargeHMAC(ResourceOwner owner); -extern void ResourceOwnerRememberHMAC(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetHMAC(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 402a6617a98..8a14175f406 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2227,8 +2227,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.2 --------------AF226508B1039DE40796DE51 Content-Type: text/x-patch; charset=UTF-8; name="v10-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v10-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v6 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1236 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 844 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..8e5b86a0e5b 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..9f9db426274 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..1db1e7cf14b 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +614,48 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +672,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +696,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +755,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +849,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +900,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..3ded16f6fc8 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2345,3 +2363,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..6421f1de5cd 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -77,7 +97,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +126,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -207,10 +226,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 943142ced8c..2df8d3cbefa 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2127,8 +2127,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.29.2 --------------F30A5443A5D524A5CBC51AE7 Content-Type: text/x-patch; charset=UTF-8; name="v6-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v6-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v6 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1236 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 844 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index b0789a5fb80..8e5b86a0e5b 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index ae82b4bdc0e..31e8860fb53 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -895,7 +916,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1162,7 +1183,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1241,3 +1262,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 20e50247ea4..de39c844d49 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 86fdfc454d2..9f9db426274 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index cc04b5b4bef..ecc9296d2d3 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (useResOwner) - ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); plan->refcount++; if (useResOwner) ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), true); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 10f15f6a357..1db1e7cf14b 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, true); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -704,16 +614,48 @@ void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { ResourceOwner save; - Datum foundres; save = CurrentResourceOwner; CurrentResourceOwner = owner; - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); + + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); - ReleaseCachedPlan(res, true); + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; + + /* + * pass 'false' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, false); + } } + CurrentResourceOwner = save; } @@ -730,19 +672,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -758,18 +696,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -827,11 +755,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -922,44 +849,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1011,424 +900,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index ae16c3ed7d6..3ded16f6fc8 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2345,3 +2363,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 551ec392b60..6421f1de5cd 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -27,7 +27,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -60,6 +59,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -77,7 +97,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -106,8 +126,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -207,10 +226,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index 79d96e5ed03..964af79b30d 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 943142ced8c..2df8d3cbefa 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2127,8 +2127,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.29.2 --------------F30A5443A5D524A5CBC51AE7 Content-Type: text/x-patch; charset=UTF-8; name="v6-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v6-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
* [PATCH v7 2/3] Make resowners more easily extensible. @ 2021-01-20 21:16 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 113+ messages in thread From: Heikki Linnakangas @ 2021-01-20 21:16 UTC (permalink / raw) Use a single array and hash, instead of one for each object kind. --- src/backend/access/common/tupdesc.c | 42 +- src/backend/jit/jit.c | 2 - src/backend/jit/llvm/llvmjit.c | 46 +- src/backend/storage/buffer/bufmgr.c | 45 +- src/backend/storage/buffer/localbuf.c | 4 +- src/backend/storage/file/fd.c | 44 +- src/backend/storage/ipc/dsm.c | 44 +- src/backend/storage/lmgr/lock.c | 2 +- src/backend/utils/cache/catcache.c | 74 +- src/backend/utils/cache/plancache.c | 45 +- src/backend/utils/cache/relcache.c | 41 +- src/backend/utils/resowner/README | 19 +- src/backend/utils/resowner/resowner.c | 1233 +++++++------------------ src/backend/utils/time/snapmgr.c | 38 +- src/common/cryptohash_openssl.c | 49 +- src/include/storage/buf_internals.h | 9 + src/include/utils/catcache.h | 3 - src/include/utils/plancache.h | 2 + src/include/utils/resowner.h | 45 +- src/include/utils/resowner_private.h | 105 --- src/tools/pgindent/typedefs.list | 4 +- 21 files changed, 841 insertions(+), 1055 deletions(-) delete mode 100644 src/include/utils/resowner_private.h diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 902f59440cd..2bf0c121fda 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -29,9 +29,27 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/datum.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" +/* ResourceOwner callbacks to hold tupledesc references */ +static void ResOwnerReleaseTupleDesc(Datum res); +static void ResOwnerPrintTupleDescLeakWarning(Datum res); + +static ResourceOwnerFuncs tupdesc_resowner_funcs = +{ + /* relcache references */ + .name = "tupdesc reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseTupleDesc, + .PrintLeakWarning = ResOwnerPrintTupleDescLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberTupleDesc(owner, tupdesc) \ + ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) +#define ResourceOwnerForgetTupleDesc(owner, tupdesc) \ + ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_funcs) /* * CreateTemplateTupleDesc @@ -376,7 +394,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc) { Assert(tupdesc->tdrefcount >= 0); - ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); tupdesc->tdrefcount++; ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); } @@ -925,3 +943,23 @@ BuildDescFromLists(List *names, List *types, List *typmods, List *collations) return desc; } + + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseTupleDesc(Datum res) +{ + DecrTupleDescRefCount((TupleDesc) DatumGetPointer(res)); +} + +static void +ResOwnerPrintTupleDescLeakWarning(Datum res) +{ + TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res); + + elog(WARNING, + "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", + tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); +} diff --git a/src/backend/jit/jit.c b/src/backend/jit/jit.c index 2da300e000d..1aa04d173b4 100644 --- a/src/backend/jit/jit.c +++ b/src/backend/jit/jit.c @@ -26,7 +26,6 @@ #include "jit/jit.h" #include "miscadmin.h" #include "utils/fmgrprotos.h" -#include "utils/resowner_private.h" /* GUCs */ bool jit_enabled = true; @@ -140,7 +139,6 @@ jit_release_context(JitContext *context) if (provider_successfully_loaded) provider.release_context(context); - ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context)); pfree(context); } diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 98a27f08bfd..45eb9cc2ff2 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -40,7 +40,7 @@ #include "portability/instr_time.h" #include "storage/ipc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Handle of a module emitted via ORC JIT */ typedef struct LLVMJitHandle @@ -121,8 +121,26 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm); static char *llvm_error_message(LLVMErrorRef error); #endif /* LLVM_VERSION_MAJOR > 11 */ -PG_MODULE_MAGIC; +/* ResourceOwner callbacks to hold JitContexts */ +static void ResOwnerReleaseJitContext(Datum res); +static void ResOwnerPrintJitContextLeakWarning(Datum res); + +static ResourceOwnerFuncs jit_resowner_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitContext, + .PrintLeakWarning = ResOwnerPrintJitContextLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberJIT(owner, handle) \ + ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_funcs) +#define ResourceOwnerForgetJIT(owner, handle) \ + ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_funcs) +PG_MODULE_MAGIC; /* * Initialize LLVM JIT provider. @@ -151,7 +169,7 @@ llvm_create_context(int jitFlags) llvm_session_initialize(); - ResourceOwnerEnlargeJIT(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); context = MemoryContextAllocZero(TopMemoryContext, sizeof(LLVMJitContext)); @@ -159,7 +177,7 @@ llvm_create_context(int jitFlags) /* ensure cleanup */ context->base.resowner = CurrentResourceOwner; - ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); + ResourceOwnerRememberJIT(CurrentResourceOwner, context); return context; } @@ -221,6 +239,8 @@ llvm_release_context(JitContext *context) pfree(jit_handle); } + + ResourceOwnerForgetJIT(context->resowner, context); } /* @@ -1231,3 +1251,21 @@ llvm_error_message(LLVMErrorRef error) } #endif /* LLVM_VERSION_MAJOR > 11 */ + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseJitContext(Datum res) +{ + jit_release_context((JitContext *) PointerGetDatum(res)); +} + +static void +ResOwnerPrintJitContextLeakWarning(Datum res) +{ + /* XXX: We used to not print these. Was that intentional? */ + JitContext *context = (JitContext *) PointerGetDatum(res); + + elog(WARNING, "JIT context leak: context %p still referenced", context); +} diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index cde869e7d64..82e39de19f9 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -52,7 +52,7 @@ #include "utils/memdebug.h" #include "utils/ps_status.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/timestamp.h" @@ -206,6 +206,18 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move static inline int32 GetPrivateRefCount(Buffer buffer); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); +/* ResourceOwner callbacks to hold buffer pins */ +static void ResOwnerReleaseBuffer(Datum res); +static void ResOwnerPrintBufferLeakWarning(Datum res); + +ResourceOwnerFuncs buffer_resowner_funcs = +{ + .name = "buffer", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseBuffer, + .PrintLeakWarning = ResOwnerPrintBufferLeakWarning +}; + /* * Ensure that the PrivateRefCountArray has sufficient space to store one more * entry. This has to be called before using NewPrivateRefCountEntry() to fill @@ -1094,7 +1106,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, * pin. */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1604,7 +1616,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ref = GetPrivateRefCountEntry(b, true); @@ -1687,7 +1699,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * * As this function is called with the spinlock held, the caller has to * previously call ReservePrivateRefCountEntry() and - * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + * ResourceOwnerEnlarge(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -2423,7 +2435,7 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3496,7 +3508,7 @@ FlushRelationBuffers(Relation rel) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3591,7 +3603,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3646,7 +3658,7 @@ FlushDatabaseBuffers(Oid dbid) /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && @@ -3729,7 +3741,7 @@ void IncrBufferRefCount(Buffer buffer) { Assert(BufferIsPinned(buffer)); - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); if (BufferIsLocal(buffer)) LocalRefCount[-buffer - 1]++; else @@ -4803,3 +4815,18 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseBuffer(Datum res) +{ + ReleaseBuffer(DatumGetInt32(res)); +} + +static void +ResOwnerPrintBufferLeakWarning(Datum res) +{ + PrintBufferLeakWarning(DatumGetInt32(res)); +} diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index f7c15ea8a44..3f22d7127b9 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -22,7 +22,7 @@ #include "storage/bufmgr.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /*#define LBDEBUG*/ @@ -124,7 +124,7 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, InitLocalBuffers(); /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index b58502837aa..1172ea245e0 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -97,7 +97,7 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "utils/guc.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ #if defined(HAVE_SYNC_FILE_RANGE) @@ -340,6 +340,24 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel); static int fsync_parent_path(const char *fname, int elevel); +/* ResourceOwner callbacks to hold virtual file descriptors */ +static void ResOwnerReleaseFile(Datum res); +static void ResOwnerPrintFileLeakWarning(Datum res); + +static ResourceOwnerFuncs file_resowner_funcs = +{ + .name = "File", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseFile, + .PrintLeakWarning = ResOwnerPrintFileLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberFile(owner, file) \ + ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_funcs) +#define ResourceOwnerForgetFile(owner, file) \ + ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_funcs) + /* * pg_fsync --- do fsync with or without writethrough */ @@ -1430,7 +1448,7 @@ ReportTemporaryFileUsage(const char *path, off_t size) /* * Called to register a temporary file for automatic close. - * ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called + * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called * before the file was opened. */ static void @@ -1612,7 +1630,7 @@ OpenTemporaryFile(bool interXact) * open it, if we'll be registering it below. */ if (!interXact) - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * If some temp tablespace(s) have been given to us, try to use the next @@ -1742,7 +1760,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* * Open the file. Note: we don't use O_EXCL, in case there is an orphaned @@ -1780,7 +1798,7 @@ PathNameOpenTemporaryFile(const char *path, int mode) { File file; - ResourceOwnerEnlargeFiles(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); file = PathNameOpenFile(path, mode | PG_BINARY); @@ -3700,3 +3718,19 @@ pg_pwritev_with_retry(int fd, const struct iovec *iov, int iovcnt, off_t offset) return sum; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseFile(Datum res) +{ + FileClose((File) DatumGetInt32(res)); +} + +static void +ResOwnerPrintFileLeakWarning(Datum res) +{ + elog(WARNING, "temporary file leak: File %d still referenced", + DatumGetInt32(res)); +} diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index b461a5f7e96..77b6e58fd9d 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -37,13 +37,15 @@ #include "miscadmin.h" #include "port/pg_bitutils.h" #include "storage/dsm.h" +#include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" #include "storage/pg_shmem.h" +#include "storage/shmem.h" #include "utils/freepage.h" #include "utils/guc.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 @@ -139,6 +141,25 @@ static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static void *dsm_control_impl_private = NULL; + +/* ResourceOwner callbacks to hold DSM segments */ +static void ResOwnerReleaseDSM(Datum res); +static void ResOwnerPrintDSMLeakWarning(Datum res); + +static ResourceOwnerFuncs dsm_resowner_funcs = +{ + .name = "dynamic shared memory segment", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseDSM, + .PrintLeakWarning = ResOwnerPrintDSMLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberDSM(owner, seg) \ + ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_funcs) +#define ResourceOwnerForgetDSM(owner, seg) \ + ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_funcs) + /* * Start up the dynamic shared memory system. * @@ -900,7 +921,7 @@ void dsm_unpin_mapping(dsm_segment *seg) { Assert(seg->resowner == NULL); - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg->resowner = CurrentResourceOwner; ResourceOwnerRememberDSM(seg->resowner, seg); } @@ -1167,7 +1188,7 @@ dsm_create_descriptor(void) dsm_segment *seg; if (CurrentResourceOwner) - ResourceOwnerEnlargeDSMs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); @@ -1246,3 +1267,20 @@ is_main_region_dsm_handle(dsm_handle handle) { return handle & 1; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseDSM(Datum res) +{ + dsm_detach((dsm_segment *) DatumGetPointer(res)); +} +static void +ResOwnerPrintDSMLeakWarning(Datum res) +{ + dsm_segment *seg = (dsm_segment *) res; + + elog(WARNING, "dynamic shared memory leak: segment %u still referenced", + dsm_segment_handle(seg)); +} diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 108b4d90238..719612da04d 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -47,7 +47,7 @@ #include "storage/standby.h" #include "utils/memutils.h" #include "utils/ps_status.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" /* This configuration variable is used to set the lock table size */ diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index 2630c800b05..1d86f543f4a 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,12 +31,13 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/syscache.h" @@ -104,6 +105,42 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, * internal support functions */ +/* ResourceOwner callbacks to hold catcache references */ + +static void ResOwnerReleaseCatCache(Datum res); +static void ResOwnerPrintCatCacheLeakWarning(Datum res); +static void ResOwnerReleaseCatCacheList(Datum res); +static void ResOwnerPrintCatCacheListLeakWarning(Datum res); + +static ResourceOwnerFuncs catcache_resowner_funcs = +{ + /* catcache references */ + .name = "catcache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCache, + .PrintLeakWarning = ResOwnerPrintCatCacheLeakWarning +}; + +static ResourceOwnerFuncs catlistref_resowner_funcs = +{ + /* catcache-list pins */ + .name = "catcache list reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCatCacheList, + .PrintLeakWarning = ResOwnerPrintCatCacheListLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCatCacheRef(owner, tuple) \ + ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerForgetCatCacheRef(owner, tuple) \ + ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_funcs) +#define ResourceOwnerRememberCatCacheListRef(owner, list) \ + ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_funcs) +#define ResourceOwnerForgetCatCacheListRef(owner, list) \ + ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_funcs) + + /* * Hash and equality functions for system types that are used as cache key * fields. In some cases, we just call the regular SQL-callable functions for @@ -1270,7 +1307,7 @@ SearchCatCacheInternal(CatCache *cache, */ if (!ct->negative) { - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); @@ -1371,7 +1408,7 @@ SearchCatCacheMiss(CatCache *cache, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ - ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); ct->refcount++; ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); break; /* assume only one match */ @@ -1583,7 +1620,7 @@ SearchCatCacheList(CatCache *cache, dlist_move_head(&cache->cc_lists, &cl->cache_elem); /* Bump the list's refcount and return it */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); cl->refcount++; ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); @@ -1695,7 +1732,7 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); /* Make sure the resource owner has room to remember this entry. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); @@ -2069,14 +2106,19 @@ PrepareToInvalidateCacheTuple(Relation relation, } } - /* - * Subroutines for warning about reference leaks. These are exported so - * that resowner.c can call them. + * ResourceOwner callbacks */ -void -PrintCatCacheLeakWarning(HeapTuple tuple) +static void +ResOwnerReleaseCatCache(Datum res) +{ + ReleaseCatCache((HeapTuple) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheLeakWarning(Datum res) { + HeapTuple tuple = (HeapTuple) DatumGetPointer(res); CatCTup *ct = (CatCTup *) (((char *) tuple) - offsetof(CatCTup, tuple)); @@ -2090,9 +2132,17 @@ PrintCatCacheLeakWarning(HeapTuple tuple) ct->refcount); } -void -PrintCatCacheListLeakWarning(CatCList *list) +static void +ResOwnerReleaseCatCacheList(Datum res) { + ReleaseCatCacheList((CatCList *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCatCacheListLeakWarning(Datum res) +{ + CatCList *list = (CatCList *) DatumGetPointer(res); + elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", list->my_cache->cc_relname, list->my_cache->id, list, list->refcount); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 1a0950489d7..b5773ba2d49 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -69,7 +69,7 @@ #include "tcop/utility.h" #include "utils/inval.h" #include "utils/memutils.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/rls.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -115,6 +115,26 @@ static void PlanCacheRelCallback(Datum arg, Oid relid); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); +/* ResourceOwner callbacks to track plancache references */ +static void ResOwnerReleaseCachedPlan(Datum res); +static void ResOwnerPrintPlanCacheLeakWarning(Datum res); + +/* this is exported for ResourceOwnerReleaseAllPlanCacheRefs() */ +ResourceOwnerFuncs planref_resowner_funcs = +{ + .name = "plancache reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseCachedPlan, + .PrintLeakWarning = ResOwnerPrintPlanCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberPlanCacheRef(owner, plan) \ + ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_funcs) +#define ResourceOwnerForgetPlanCacheRef(owner, plan) \ + ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_funcs) + + /* GUC parameter */ int plan_cache_mode; @@ -1229,7 +1249,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, /* Flag the plan as in use by caller */ if (owner) - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; if (owner) ResourceOwnerRememberPlanCacheRef(owner, plan); @@ -1392,7 +1412,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -1451,7 +1471,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan, /* It's still good. Bump refcount if requested. */ if (owner) { - ResourceOwnerEnlargePlanCacheRefs(owner); + ResourceOwnerEnlarge(owner); plan->refcount++; ResourceOwnerRememberPlanCacheRef(owner, plan); } @@ -2205,3 +2225,20 @@ ResetPlanCache(void) cexpr->is_valid = false; } } + +/* + * ResourceOwner callbacks + */ + +static void +ResOwnerReleaseCachedPlan(Datum res) +{ + ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), CurrentResourceOwner); +} + +static void +ResOwnerPrintPlanCacheLeakWarning(Datum res) +{ + elog(WARNING, "plancache reference leak: plan %p not closed", + DatumGetPointer(res)); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 7ef510cd01b..6ca5b50552b 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -78,13 +78,14 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/catcache.h" #include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/relmapper.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -2078,6 +2079,24 @@ RelationIdGetRelation(Oid relationId) * ---------------------------------------------------------------- */ +/* ResourceOwner callbacks to track relcache references */ +static void ResOwnerReleaseRelation(Datum res); +static void ResOwnerPrintRelCacheLeakWarning(Datum res); + +static ResourceOwnerFuncs relref_resowner_funcs = +{ + .name = "relcache reference", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseRelation, + .PrintLeakWarning = ResOwnerPrintRelCacheLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberRelationRef(owner, rel) \ + ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_funcs) +#define ResourceOwnerForgetRelationRef(owner, rel) \ + ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_funcs) + /* * RelationIncrementReferenceCount * Increments relation reference count. @@ -2089,7 +2108,7 @@ RelationIdGetRelation(Oid relationId) void RelationIncrementReferenceCount(Relation rel) { - ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); rel->rd_refcnt += 1; if (!IsBootstrapProcessingMode()) ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); @@ -6417,3 +6436,21 @@ unlink_initfile(const char *initfilename, int elevel) initfilename))); } } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerPrintRelCacheLeakWarning(Datum res) +{ + Relation rel = (Relation) res; + + elog(WARNING, "relcache reference leak: relation \"%s\" not closed", + RelationGetRelationName(rel)); +} + +static void +ResOwnerReleaseRelation(Datum res) +{ + RelationClose((Relation) res); +} diff --git a/src/backend/utils/resowner/README b/src/backend/utils/resowner/README index 2998f6bb362..4ed13afa101 100644 --- a/src/backend/utils/resowner/README +++ b/src/backend/utils/resowner/README @@ -60,13 +60,18 @@ subtransaction or portal. Therefore, the "release" operation on a child ResourceOwner transfers lock ownership to the parent instead of actually releasing the lock, if isCommit is true. -Currently, ResourceOwners contain direct support for recording ownership of -buffer pins, lmgr locks, and catcache, relcache, plancache, tupdesc, and -snapshot references. Other objects can be associated with a ResourceOwner by -recording the address of the owning ResourceOwner in such an object. There is -an API for other modules to get control during ResourceOwner release, so that -they can scan their own data structures to find the objects that need to be -deleted. +ResourceOwner can record ownership of many different kinds of resources. +As of this writing, it's used internally for buffer pins, lmgr locks, and +catcache, relcache, plancache, tupdesc, snapshot, DSM and JIT context +references. ResourceOwner treats all resources the same, and extensions +can define new kinds of resources by filling in a ResourceOwnerFuncs +struct with a suitable callback functions. + +There is also an API for other modules to get control during ResourceOwner +release, so that they can scan their own data structures to find the objects +that need to be deleted. This used to be the only way to register new kinds +of objects with a resource owner; nowadays it easier to write custom +ResourceOwnerFuncs callbacks. Whenever we are inside a transaction, the global variable CurrentResourceOwner shows which resource owner should be assigned diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index a171df573ce..d62b1495dd3 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -20,75 +20,45 @@ */ #include "postgres.h" -#include "common/cryptohash.h" #include "common/hashfn.h" -#include "jit/jit.h" -#include "storage/bufmgr.h" #include "storage/ipc.h" #include "storage/predicate.h" #include "storage/proc.h" #include "utils/memutils.h" -#include "utils/rel.h" -#include "utils/resowner_private.h" -#include "utils/snapmgr.h" - - -/* - * All resource IDs managed by this code are required to fit into a Datum, - * which is fine since they are generally pointers or integers. - * - * Provide Datum conversion macros for a couple of things that are really - * just "int". - */ -#define FileGetDatum(file) Int32GetDatum(file) -#define DatumGetFile(datum) ((File) DatumGetInt32(datum)) -#define BufferGetDatum(buffer) Int32GetDatum(buffer) -#define DatumGetBuffer(datum) ((Buffer) DatumGetInt32(datum)) +#include "utils/plancache.h" +#include "utils/resowner.h" /* - * ResourceArray is a common structure for storing all types of resource IDs. - * - * We manage small sets of resource IDs by keeping them in a simple array: - * itemsarr[k] holds an ID, for 0 <= k < nitems <= maxitems = capacity. - * - * If a set grows large, we switch over to using open-addressing hashing. - * Then, itemsarr[] is a hash table of "capacity" slots, with each - * slot holding either an ID or "invalidval". nitems is the number of valid - * items present; if it would exceed maxitems, we enlarge the array and - * re-hash. In this mode, maxitems should be rather less than capacity so - * that we don't waste too much time searching for empty slots. + * ResourceElem represents a reference associated with a resource owner. * - * In either mode, lastidx remembers the location of the last item inserted - * or returned by GetAny; this speeds up searches in ResourceArrayRemove. + * All objects managed by this code are required to fit into a Datum, + * which is fine since they are generally pointers or integers. */ -typedef struct ResourceArray +typedef struct ResourceElem { - Datum *itemsarr; /* buffer for storing values */ - Datum invalidval; /* value that is considered invalid */ - uint32 capacity; /* allocated length of itemsarr[] */ - uint32 nitems; /* how many items are stored in items array */ - uint32 maxitems; /* current limit on nitems before enlarging */ - uint32 lastidx; /* index of last item returned by GetAny */ -} ResourceArray; + Datum item; + ResourceOwnerFuncs *kind; +} ResourceElem; /* - * Initially allocated size of a ResourceArray. Must be power of two since - * we'll use (arraysize - 1) as mask for hashing. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define RESARRAY_INIT_SIZE 16 +#define RESOWNER_ARRAY_SIZE 8 /* - * When to switch to hashing vs. simple array logic in a ResourceArray. + * Initially allocated size of a ResourceOwner's hash. Must be power of two since + * we'll use (capacity - 1) as mask for hashing. */ -#define RESARRAY_MAX_ARRAY 64 -#define RESARRAY_IS_ARRAY(resarr) ((resarr)->capacity <= RESARRAY_MAX_ARRAY) +#define RESOWNER_HASH_INIT_SIZE 32 /* - * How many items may be stored in a resource array of given capacity. + * How many items may be stored in a hash of given capacity. * When this number is reached, we must resize. */ -#define RESARRAY_MAX_ITEMS(capacity) \ - ((capacity) <= RESARRAY_MAX_ARRAY ? (capacity) : (capacity)/4 * 3) +#define RESOWNER_HASH_MAX_ITEMS(capacity) ((capacity)/4 * 3) + +StaticAssertDecl(RESOWNER_HASH_MAX_ITEMS(RESOWNER_HASH_INIT_SIZE) > RESOWNER_ARRAY_SIZE, + "initial hash size too small compared to array size"); /* * To speed up bulk releasing or reassigning locks from a resource owner to @@ -118,23 +88,33 @@ typedef struct ResourceOwnerData ResourceOwner nextchild; /* next child of same parent */ const char *name; /* name (just for debugging) */ - /* We have built-in support for remembering: */ - ResourceArray bufferarr; /* owned buffers */ - ResourceArray catrefarr; /* catcache references */ - ResourceArray catlistrefarr; /* catcache-list pins */ - ResourceArray relrefarr; /* relcache references */ - ResourceArray planrefarr; /* plancache references */ - ResourceArray tupdescarr; /* tupdesc references */ - ResourceArray snapshotarr; /* snapshot references */ - ResourceArray filearr; /* open temporary files */ - ResourceArray dsmarr; /* dynamic shmem segments */ - ResourceArray jitarr; /* JIT contexts */ - ResourceArray cryptohasharr; /* cryptohash contexts */ + /* + * These structs keep track of the objects registered with this owner. + * + * We manage a small set of references by keeping them in a simple array. + * When the array gets full, all the elements in the array are moved to a + * hash table. This way, the array always contains a few most recently + * remembered references. To find a particular reference, you need to + * search both the array and the hash table. + */ + ResourceElem arr[RESOWNER_ARRAY_SIZE]; + uint32 narr; /* how many items are stored in the array */ + + /* + * The hash table. Uses open-addressing. 'nhash' is the number of items + * present; if it would exceed 'grow_at', we enlarge it and re-hash. + * 'grow_at' should be rather less than 'capacity' so that we don't waste + * too much time searching for empty slots. + */ + ResourceElem *hash; + uint32 nhash; /* how many items are stored in the hash */ + uint32 capacity; /* allocated length of hash[] */ + uint32 grow_at; /* grow hash when reach this */ /* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */ int nlocks; /* number of owned locks */ LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */ -} ResourceOwnerData; +} ResourceOwnerData; /***************************************************************************** @@ -146,6 +126,18 @@ ResourceOwner CurTransactionResourceOwner = NULL; ResourceOwner TopTransactionResourceOwner = NULL; ResourceOwner AuxProcessResourceOwner = NULL; +/* #define RESOWNER_STATS */ +/* #define RESOWNER_TRACE */ + +#ifdef RESOWNER_STATS +static int narray_lookups = 0; +static int nhash_lookups = 0; +#endif + +#ifdef RESOWNER_TRACE +static int resowner_trace_counter = 0; +#endif + /* * List of add-on callbacks for resource releasing */ @@ -160,296 +152,315 @@ static ResourceReleaseCallbackItem *ResourceRelease_callbacks = NULL; /* Internal routines */ -static void ResourceArrayInit(ResourceArray *resarr, Datum invalidval); -static void ResourceArrayEnlarge(ResourceArray *resarr); -static void ResourceArrayAdd(ResourceArray *resarr, Datum value); -static bool ResourceArrayRemove(ResourceArray *resarr, Datum value); -static bool ResourceArrayGetAny(ResourceArray *resarr, Datum *value); -static void ResourceArrayFree(ResourceArray *resarr); static void ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); static void ReleaseAuxProcessResourcesCallback(int code, Datum arg); -static void PrintRelCacheLeakWarning(Relation rel); -static void PrintPlanCacheLeakWarning(CachedPlan *plan); -static void PrintTupleDescLeakWarning(TupleDesc tupdesc); -static void PrintSnapshotLeakWarning(Snapshot snapshot); -static void PrintFileLeakWarning(File file); -static void PrintDSMLeakWarning(dsm_segment *seg); -static void PrintCryptoHashLeakWarning(Datum handle); /***************************************************************************** * INTERNAL ROUTINES * *****************************************************************************/ +static inline uint32 +hash_resource_elem(Datum value, ResourceOwnerFuncs *kind) +{ + Datum data[2]; + + data[0] = value; + data[1] = PointerGetDatum(kind); + + return hash_bytes((unsigned char *) &data, 2 * SIZEOF_DATUM); +} -/* - * Initialize a ResourceArray - */ static void -ResourceArrayInit(ResourceArray *resarr, Datum invalidval) +ResourceOwnerAddToHash(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - /* Assert it's empty */ - Assert(resarr->itemsarr == NULL); - Assert(resarr->capacity == 0); - Assert(resarr->nitems == 0); - Assert(resarr->maxitems == 0); - /* Remember the appropriate "invalid" value */ - resarr->invalidval = invalidval; - /* We don't allocate any storage until needed */ + /* Insert into first free slot at or after hash location. */ + uint32 mask = owner->capacity - 1; + uint32 idx; + + idx = hash_resource_elem(value, kind) & mask; + for (;;) + { + if (owner->hash[idx].kind == NULL) + break; + idx = (idx + 1) & mask; + } + owner->hash[idx].item = value; + owner->hash[idx].kind = kind; + owner->nhash++; } /* - * Make sure there is room for at least one more resource in an array. - * - * This is separate from actually inserting a resource because if we run out - * of memory, it's critical to do so *before* acquiring the resource. + * Call the ReleaseResource callback on entries with given 'phase'. */ static void -ResourceArrayEnlarge(ResourceArray *resarr) +ResourceOwnerReleaseAll(ResourceOwner owner, ResourceReleasePhase phase, + bool printLeakWarnings) { - uint32 i, - oldcap, - newcap; - Datum *olditemsarr; - Datum *newitemsarr; - - if (resarr->nitems < resarr->maxitems) - return; /* no work needed */ + bool found; + int capacity; - olditemsarr = resarr->itemsarr; - oldcap = resarr->capacity; - - /* Double the capacity of the array (capacity must stay a power of 2!) */ - newcap = (oldcap > 0) ? oldcap * 2 : RESARRAY_INIT_SIZE; - newitemsarr = (Datum *) MemoryContextAlloc(TopMemoryContext, - newcap * sizeof(Datum)); - for (i = 0; i < newcap; i++) - newitemsarr[i] = resarr->invalidval; + /* First handle all the entries in the array. */ + do + { + found = false; + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind->phase == phase) + { + Datum value = owner->arr[i].item; + ResourceOwnerFuncs *kind = owner->arr[i].kind; - /* We assume we can't fail below this point, so OK to scribble on resarr */ - resarr->itemsarr = newitemsarr; - resarr->capacity = newcap; - resarr->maxitems = RESARRAY_MAX_ITEMS(newcap); - resarr->nitems = 0; + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + found = true; + } + } - if (olditemsarr != NULL) - { /* - * Transfer any pre-existing entries into the new array; they don't - * necessarily go where they were before, so this simple logic is the - * best way. Note that if we were managing the set as a simple array, - * the entries after nitems are garbage, but that shouldn't matter - * because we won't get here unless nitems was equal to oldcap. + * If any resources were released, check again because some of the + * elements might have been moved by the callbacks. We don't want to + * miss them. */ - for (i = 0; i < oldcap; i++) + } while (found && owner->narr > 0); + + /* Ok, the array has now been handled. Then the hash */ + do + { + capacity = owner->capacity; + for (int idx = 0; idx < capacity; idx++) { - if (olditemsarr[i] != resarr->invalidval) - ResourceArrayAdd(resarr, olditemsarr[i]); + while (owner->hash[idx].kind != NULL && + owner->hash[idx].kind->phase == phase) + { + Datum value = owner->hash[idx].item; + ResourceOwnerFuncs *kind = owner->hash[idx].kind; + + if (printLeakWarnings) + kind->PrintLeakWarning(value); + kind->ReleaseResource(value); + + /* + * If the resource is remembered more than once in this + * resource owner, the ReleaseResource callback might've + * released a different copy of it. Because of that, loop to + * check the same index again. + */ + } } - /* And release old array. */ - pfree(olditemsarr); - } - - Assert(resarr->nitems < resarr->maxitems); + /* + * It's possible that the callbacks acquired more resources, causing + * the hash table to grow and the existing entries to be moved around. + * If that happened, scan the hash table again, so that we don't miss + * entries that were moved. (XXX: I'm not sure if any of the callbacks + * actually do that, but this is cheap to check, and better safe than + * sorry.) + */ + Assert(owner->capacity >= capacity); + } while (capacity != owner->capacity); } + +/***************************************************************************** + * EXPORTED ROUTINES * + *****************************************************************************/ + + /* - * Add a resource to ResourceArray + * ResourceOwnerCreate + * Create an empty ResourceOwner. * - * Caller must have previously done ResourceArrayEnlarge() + * All ResourceOwner objects are kept in TopMemoryContext, since they should + * only be freed explicitly. */ -static void -ResourceArrayAdd(ResourceArray *resarr, Datum value) +ResourceOwner +ResourceOwnerCreate(ResourceOwner parent, const char *name) { - uint32 idx; + ResourceOwner owner; - Assert(value != resarr->invalidval); - Assert(resarr->nitems < resarr->maxitems); + owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, + sizeof(ResourceOwnerData)); + owner->name = name; - if (RESARRAY_IS_ARRAY(resarr)) + if (parent) { - /* Append to linear array. */ - idx = resarr->nitems; + owner->parent = parent; + owner->nextchild = parent->firstchild; + parent->firstchild = owner; } - else - { - /* Insert into first free slot at or after hash location. */ - uint32 mask = resarr->capacity - 1; - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (;;) - { - if (resarr->itemsarr[idx] == resarr->invalidval) - break; - idx = (idx + 1) & mask; - } - } - resarr->lastidx = idx; - resarr->itemsarr[idx] = value; - resarr->nitems++; +#ifdef RESOWNER_TRACE + elog(LOG, "CREATE %d: %p %s", + resowner_trace_counter++, owner, name); +#endif + + return owner; } /* - * Remove a resource from ResourceArray - * - * Returns true on success, false if resource was not found. + * Make sure there is room for at least one more resource in an array. * - * Note: if same resource ID appears more than once, one instance is removed. + * This is separate from actually inserting a resource because if we run out + * of memory, it's critical to do so *before* acquiring the resource. */ -static bool -ResourceArrayRemove(ResourceArray *resarr, Datum value) +void +ResourceOwnerEnlarge(ResourceOwner owner) { - uint32 i, - idx, - lastidx = resarr->lastidx; - - Assert(value != resarr->invalidval); + if (owner->narr < RESOWNER_ARRAY_SIZE) + return; /* no work needed */ - /* Search through all items, but try lastidx first. */ - if (RESARRAY_IS_ARRAY(resarr)) + /* Is there space in the hash? If not, enlarge it. */ + if (owner->narr + owner->nhash >= owner->grow_at) { - if (lastidx < resarr->nitems && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; - } - for (i = 0; i < resarr->nitems; i++) + uint32 i, + oldcap, + newcap; + ResourceElem *oldhash; + ResourceElem *newhash; + + oldhash = owner->hash; + oldcap = owner->capacity; + + /* Double the capacity (it must stay a power of 2!) */ + newcap = (oldcap > 0) ? oldcap * 2 : RESOWNER_HASH_INIT_SIZE; + newhash = (ResourceElem *) MemoryContextAllocZero(TopMemoryContext, + newcap * sizeof(ResourceElem)); + + /* + * We assume we can't fail below this point, so OK to scribble on the + * owner + */ + owner->hash = newhash; + owner->capacity = newcap; + owner->grow_at = RESOWNER_HASH_MAX_ITEMS(newcap); + owner->nhash = 0; + + if (oldhash != NULL) { - if (resarr->itemsarr[i] == value) + /* + * Transfer any pre-existing entries into the new hash table; they + * don't necessarily go where they were before, so this simple + * logic is the best way. + */ + for (i = 0; i < oldcap; i++) { - resarr->itemsarr[i] = resarr->itemsarr[resarr->nitems - 1]; - resarr->nitems--; - /* Update lastidx to make reverse-order removals fast. */ - resarr->lastidx = resarr->nitems - 1; - return true; + if (oldhash[i].kind != NULL) + ResourceOwnerAddToHash(owner, oldhash[i].item, oldhash[i].kind); } + + /* And release old hash table. */ + pfree(oldhash); } } - else - { - uint32 mask = resarr->capacity - 1; - if (lastidx < resarr->capacity && - resarr->itemsarr[lastidx] == value) - { - resarr->itemsarr[lastidx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = DatumGetUInt32(hash_any((void *) &value, sizeof(value))) & mask; - for (i = 0; i < resarr->capacity; i++) - { - if (resarr->itemsarr[idx] == value) - { - resarr->itemsarr[idx] = resarr->invalidval; - resarr->nitems--; - return true; - } - idx = (idx + 1) & mask; - } + /* Move items from the array to the hash */ + Assert(owner->narr == RESOWNER_ARRAY_SIZE); + for (int i = 0; i < owner->narr; i++) + { + ResourceOwnerAddToHash(owner, owner->arr[i].item, owner->arr[i].kind); } + owner->narr = 0; - return false; + Assert(owner->nhash < owner->grow_at); } /* - * Get any convenient entry in a ResourceArray. + * Remember that an object is owner by a ReourceOwner * - * "Convenient" is defined as "easy for ResourceArrayRemove to remove"; - * we help that along by setting lastidx to match. This avoids O(N^2) cost - * when removing all ResourceArray items during ResourceOwner destruction. - * - * Returns true if we found an element, or false if the array is empty. + * Caller must have previously done ResourceOwnerEnlarge() */ -static bool -ResourceArrayGetAny(ResourceArray *resarr, Datum *value) +void +ResourceOwnerRemember(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->nitems == 0) - return false; + uint32 idx; - if (RESARRAY_IS_ARRAY(resarr)) - { - /* Linear array: just return the first element. */ - resarr->lastidx = 0; - } - else - { - /* Hash: search forward from wherever we were last. */ - uint32 mask = resarr->capacity - 1; +#ifdef RESOWNER_TRACE + elog(LOG, "REMEMBER %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif - for (;;) - { - resarr->lastidx &= mask; - if (resarr->itemsarr[resarr->lastidx] != resarr->invalidval) - break; - resarr->lastidx++; - } - } + Assert(owner->narr < RESOWNER_ARRAY_SIZE); - *value = resarr->itemsarr[resarr->lastidx]; - return true; + /* Append to linear array. */ + idx = owner->narr; + owner->arr[idx].item = value; + owner->arr[idx].kind = kind; + owner->narr++; } /* - * Trash a ResourceArray (we don't care about its state after this) + * Forget that an object is owned by a ResourceOwner + * + * Returns true on success. If the resource was not found, returns false, + * and calls kind->ForgetError callback. + * + * Note: if same resource ID is associated with the ResourceOwner more than once, + * one instance is removed. */ -static void -ResourceArrayFree(ResourceArray *resarr) +void +ResourceOwnerForget(ResourceOwner owner, Datum value, ResourceOwnerFuncs *kind) { - if (resarr->itemsarr) - pfree(resarr->itemsarr); -} - + uint32 i, + idx; -/***************************************************************************** - * EXPORTED ROUTINES * - *****************************************************************************/ +#ifdef RESOWNER_TRACE + elog(LOG, "FORGET %d: owner %p value " UINT64_FORMAT ", kind: %s", + resowner_trace_counter++, owner, DatumGetUInt64(value), kind->name); +#endif + /* Search through all items, but check the array first. */ + for (i = 0; i < owner->narr; i++) + { + if (owner->arr[i].item == value && + owner->arr[i].kind == kind) + { + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; -/* - * ResourceOwnerCreate - * Create an empty ResourceOwner. - * - * All ResourceOwner objects are kept in TopMemoryContext, since they should - * only be freed explicitly. - */ -ResourceOwner -ResourceOwnerCreate(ResourceOwner parent, const char *name) -{ - ResourceOwner owner; +#ifdef RESOWNER_STATS + narray_lookups++; +#endif - owner = (ResourceOwner) MemoryContextAllocZero(TopMemoryContext, - sizeof(ResourceOwnerData)); - owner->name = name; + return; + } + } - if (parent) + /* Search hash */ + if (owner->nhash > 0) { - owner->parent = parent; - owner->nextchild = parent->firstchild; - parent->firstchild = owner; - } + uint32 mask = owner->capacity - 1; - ResourceArrayInit(&(owner->bufferarr), BufferGetDatum(InvalidBuffer)); - ResourceArrayInit(&(owner->catrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->catlistrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->relrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->planrefarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->tupdescarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->snapshotarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->filearr), FileGetDatum(-1)); - ResourceArrayInit(&(owner->dsmarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->jitarr), PointerGetDatum(NULL)); - ResourceArrayInit(&(owner->cryptohasharr), PointerGetDatum(NULL)); + idx = hash_resource_elem(value, kind) & mask; + for (i = 0; i < owner->capacity; i++) + { + if (owner->hash[idx].item == value && + owner->hash[idx].kind == kind) + { + owner->hash[idx].item = (Datum) 0; + owner->hash[idx].kind = NULL; + owner->nhash--; + +#ifdef RESOWNER_STATS + nhash_lookups++; +#endif + return; + } + idx = (idx + 1) & mask; + } + } - return owner; + /* + * Use %p to print the reference, since most objects tracked by a resource + * owner are pointers. It's a bit misleading if it's not a pointer, but + * this is a programmer error, anyway. + */ + elog(ERROR, "%s %p is not owned by resource owner %s", + kind->name, DatumGetPointer(value), owner->name); } /* @@ -486,6 +497,15 @@ ResourceOwnerRelease(ResourceOwner owner, { /* There's not currently any setup needed before recursing */ ResourceOwnerReleaseInternal(owner, phase, isCommit, isTopLevel); + +#ifdef RESOWNER_STATS + if (isTopLevel) + { + elog(LOG, "RESOWNER STATS: lookups: array %d, hash %d", narray_lookups, nhash_lookups); + narray_lookups = 0; + nhash_lookups = 0; + } +#endif } static void @@ -497,7 +517,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, ResourceOwner child; ResourceOwner save; ResourceReleaseCallbackItem *item; - Datum foundres; /* Recurse to handle descendants */ for (child = owner->firstchild; child != NULL; child = child->nextchild) @@ -513,61 +532,13 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, if (phase == RESOURCE_RELEASE_BEFORE_LOCKS) { /* - * Release buffer pins. Note that ReleaseBuffer will remove the - * buffer entry from our array, so we just have to iterate till there - * are none. + * Release all references that need to be released before the locks. * - * During a commit, there shouldn't be any remaining pins --- that - * would indicate failure to clean up the executor correctly --- so - * issue warnings. In the abort case, just clean up quietly. + * During a commit, there shouldn't be any remaining references --- + * that would indicate failure to clean up the executor correctly --- + * so issue warnings. In the abort case, just clean up quietly. */ - while (ResourceArrayGetAny(&(owner->bufferarr), &foundres)) - { - Buffer res = DatumGetBuffer(foundres); - - if (isCommit) - PrintBufferLeakWarning(res); - ReleaseBuffer(res); - } - - /* Ditto for relcache references */ - while (ResourceArrayGetAny(&(owner->relrefarr), &foundres)) - { - Relation res = (Relation) DatumGetPointer(foundres); - - if (isCommit) - PrintRelCacheLeakWarning(res); - RelationClose(res); - } - - /* Ditto for dynamic shared memory segments */ - while (ResourceArrayGetAny(&(owner->dsmarr), &foundres)) - { - dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres); - - if (isCommit) - PrintDSMLeakWarning(res); - dsm_detach(res); - } - - /* Ditto for JIT contexts */ - while (ResourceArrayGetAny(&(owner->jitarr), &foundres)) - { - JitContext *context = (JitContext *) PointerGetDatum(foundres); - - jit_release_context(context); - } - - /* Ditto for cryptohash contexts */ - while (ResourceArrayGetAny(&(owner->cryptohasharr), &foundres)) - { - pg_cryptohash_ctx *context = - (pg_cryptohash_ctx *) PointerGetDatum(foundres); - - if (isCommit) - PrintCryptoHashLeakWarning(foundres); - pg_cryptohash_free(context); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } else if (phase == RESOURCE_RELEASE_LOCKS) { @@ -576,7 +547,7 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, /* * For a top-level xact we are going to release all locks (or at * least all non-session locks), so just do a single lmgr call at - * the top of the recursion. + * the top of the recursion */ if (owner == TopTransactionResourceOwner) { @@ -620,70 +591,9 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, else if (phase == RESOURCE_RELEASE_AFTER_LOCKS) { /* - * Release catcache references. Note that ReleaseCatCache will remove - * the catref entry from our array, so we just have to iterate till - * there are none. - * - * As with buffer pins, warn if any are left at commit time. + * Release all references that need to be released after the locks. */ - while (ResourceArrayGetAny(&(owner->catrefarr), &foundres)) - { - HeapTuple res = (HeapTuple) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheLeakWarning(res); - ReleaseCatCache(res); - } - - /* Ditto for catcache lists */ - while (ResourceArrayGetAny(&(owner->catlistrefarr), &foundres)) - { - CatCList *res = (CatCList *) DatumGetPointer(foundres); - - if (isCommit) - PrintCatCacheListLeakWarning(res); - ReleaseCatCacheList(res); - } - - /* Ditto for plancache references */ - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) - { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); - - if (isCommit) - PrintPlanCacheLeakWarning(res); - ReleaseCachedPlan(res, owner); - } - - /* Ditto for tupdesc references */ - while (ResourceArrayGetAny(&(owner->tupdescarr), &foundres)) - { - TupleDesc res = (TupleDesc) DatumGetPointer(foundres); - - if (isCommit) - PrintTupleDescLeakWarning(res); - DecrTupleDescRefCount(res); - } - - /* Ditto for snapshot references */ - while (ResourceArrayGetAny(&(owner->snapshotarr), &foundres)) - { - Snapshot res = (Snapshot) DatumGetPointer(foundres); - - if (isCommit) - PrintSnapshotLeakWarning(res); - UnregisterSnapshot(res); - } - - /* Ditto for temporary files */ - while (ResourceArrayGetAny(&(owner->filearr), &foundres)) - { - File res = DatumGetFile(foundres); - - if (isCommit) - PrintFileLeakWarning(res); - FileClose(res); - } + ResourceOwnerReleaseAll(owner, phase, isCommit); } /* Let add-on modules get a chance too */ @@ -703,13 +613,42 @@ ResourceOwnerReleaseInternal(ResourceOwner owner, void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner) { - Datum foundres; + /* array first */ + for (int i = 0; i < owner->narr; i++) + { + if (owner->arr[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->arr[i].item); - while (ResourceArrayGetAny(&(owner->planrefarr), &foundres)) + owner->arr[i] = owner->arr[owner->narr - 1]; + owner->narr--; + i--; + + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } + } + + /* Then hash */ + for (int i = 0; i < owner->capacity; i++) { - CachedPlan *res = (CachedPlan *) DatumGetPointer(foundres); + if (owner->hash[i].kind == &planref_resowner_funcs) + { + CachedPlan *planref = (CachedPlan *) DatumGetPointer(owner->hash[i].item); + + owner->hash[i].item = (Datum) 0; + owner->hash[i].kind = NULL; + owner->nhash--; - ReleaseCachedPlan(res, owner); + /* + * pass 'NULL' because we already removed the entry from the + * resowner + */ + ReleaseCachedPlan(planref, NULL); + } } } @@ -726,19 +665,15 @@ ResourceOwnerDelete(ResourceOwner owner) Assert(owner != CurrentResourceOwner); /* And it better not own any resources, either */ - Assert(owner->bufferarr.nitems == 0); - Assert(owner->catrefarr.nitems == 0); - Assert(owner->catlistrefarr.nitems == 0); - Assert(owner->relrefarr.nitems == 0); - Assert(owner->planrefarr.nitems == 0); - Assert(owner->tupdescarr.nitems == 0); - Assert(owner->snapshotarr.nitems == 0); - Assert(owner->filearr.nitems == 0); - Assert(owner->dsmarr.nitems == 0); - Assert(owner->jitarr.nitems == 0); - Assert(owner->cryptohasharr.nitems == 0); + Assert(owner->narr == 0); + Assert(owner->nhash == 0); Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1); +#ifdef RESOWNER_TRACE + elog(LOG, "DELETE %d: %p %s", + resowner_trace_counter++, owner, owner->name); +#endif + /* * Delete children. The recursive call will delink the child from me, so * just iterate as long as there is a child. @@ -754,18 +689,8 @@ ResourceOwnerDelete(ResourceOwner owner) ResourceOwnerNewParent(owner, NULL); /* And free the object. */ - ResourceArrayFree(&(owner->bufferarr)); - ResourceArrayFree(&(owner->catrefarr)); - ResourceArrayFree(&(owner->catlistrefarr)); - ResourceArrayFree(&(owner->relrefarr)); - ResourceArrayFree(&(owner->planrefarr)); - ResourceArrayFree(&(owner->tupdescarr)); - ResourceArrayFree(&(owner->snapshotarr)); - ResourceArrayFree(&(owner->filearr)); - ResourceArrayFree(&(owner->dsmarr)); - ResourceArrayFree(&(owner->jitarr)); - ResourceArrayFree(&(owner->cryptohasharr)); - + if (owner->hash) + pfree(owner->hash); pfree(owner); } @@ -823,11 +748,10 @@ ResourceOwnerNewParent(ResourceOwner owner, /* * Register or deregister callback functions for resource cleanup * - * These functions are intended for use by dynamically loaded modules. - * For built-in modules we generally just hardwire the appropriate calls. - * - * Note that the callback occurs post-commit or post-abort, so the callback - * functions can only do noncritical cleanup. + * These functions can be used by dynamically loaded modules. These used + * to be the only way for an extension to register custom resource types + * with a resource owner, but nowadays it is easier to define a new + * ResourceOwnerFuncs instance with custom callbacks. */ void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg) @@ -918,44 +842,6 @@ ReleaseAuxProcessResourcesCallback(int code, Datum arg) ReleaseAuxProcessResources(isCommit); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * buffer array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeBuffers(ResourceOwner owner) -{ - /* We used to allow pinning buffers without a resowner, but no more */ - Assert(owner != NULL); - ResourceArrayEnlarge(&(owner->bufferarr)); -} - -/* - * Remember that a buffer pin is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeBuffers() - */ -void -ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer) -{ - ResourceArrayAdd(&(owner->bufferarr), BufferGetDatum(buffer)); -} - -/* - * Forget that a buffer pin is owned by a ResourceOwner - */ -void -ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer) -{ - if (!ResourceArrayRemove(&(owner->bufferarr), BufferGetDatum(buffer))) - elog(ERROR, "buffer %d is not owned by resource owner %s", - buffer, owner->name); -} - /* * Remember that a Local Lock is owned by a ResourceOwner * @@ -1007,424 +893,3 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock) elog(ERROR, "lock reference %p is not owned by resource owner %s", locallock, owner->name); } - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catrefarr)); -} - -/* - * Remember that a catcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheRefs() - */ -void -ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - ResourceArrayAdd(&(owner->catrefarr), PointerGetDatum(tuple)); -} - -/* - * Forget that a catcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple) -{ - if (!ResourceArrayRemove(&(owner->catrefarr), PointerGetDatum(tuple))) - elog(ERROR, "catcache reference %p is not owned by resource owner %s", - tuple, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * catcache-list reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->catlistrefarr)); -} - -/* - * Remember that a catcache-list reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCatCacheListRefs() - */ -void -ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - ResourceArrayAdd(&(owner->catlistrefarr), PointerGetDatum(list)); -} - -/* - * Forget that a catcache-list reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list) -{ - if (!ResourceArrayRemove(&(owner->catlistrefarr), PointerGetDatum(list))) - elog(ERROR, "catcache list reference %p is not owned by resource owner %s", - list, owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * relcache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeRelationRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->relrefarr)); -} - -/* - * Remember that a relcache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeRelationRefs() - */ -void -ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel) -{ - ResourceArrayAdd(&(owner->relrefarr), PointerGetDatum(rel)); -} - -/* - * Forget that a relcache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel) -{ - if (!ResourceArrayRemove(&(owner->relrefarr), PointerGetDatum(rel))) - elog(ERROR, "relcache reference %s is not owned by resource owner %s", - RelationGetRelationName(rel), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintRelCacheLeakWarning(Relation rel) -{ - elog(WARNING, "relcache reference leak: relation \"%s\" not closed", - RelationGetRelationName(rel)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * plancache reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->planrefarr)); -} - -/* - * Remember that a plancache reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargePlanCacheRefs() - */ -void -ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - ResourceArrayAdd(&(owner->planrefarr), PointerGetDatum(plan)); -} - -/* - * Forget that a plancache reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan) -{ - if (!ResourceArrayRemove(&(owner->planrefarr), PointerGetDatum(plan))) - elog(ERROR, "plancache reference %p is not owned by resource owner %s", - plan, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintPlanCacheLeakWarning(CachedPlan *plan) -{ - elog(WARNING, "plancache reference leak: plan %p not closed", plan); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * tupdesc reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeTupleDescs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->tupdescarr)); -} - -/* - * Remember that a tupdesc reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeTupleDescs() - */ -void -ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - ResourceArrayAdd(&(owner->tupdescarr), PointerGetDatum(tupdesc)); -} - -/* - * Forget that a tupdesc reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc) -{ - if (!ResourceArrayRemove(&(owner->tupdescarr), PointerGetDatum(tupdesc))) - elog(ERROR, "tupdesc reference %p is not owned by resource owner %s", - tupdesc, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintTupleDescLeakWarning(TupleDesc tupdesc) -{ - elog(WARNING, - "TupleDesc reference leak: TupleDesc %p (%u,%d) still referenced", - tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * snapshot reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeSnapshots(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->snapshotarr)); -} - -/* - * Remember that a snapshot reference is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeSnapshots() - */ -void -ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - ResourceArrayAdd(&(owner->snapshotarr), PointerGetDatum(snapshot)); -} - -/* - * Forget that a snapshot reference is owned by a ResourceOwner - */ -void -ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot) -{ - if (!ResourceArrayRemove(&(owner->snapshotarr), PointerGetDatum(snapshot))) - elog(ERROR, "snapshot reference %p is not owned by resource owner %s", - snapshot, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintSnapshotLeakWarning(Snapshot snapshot) -{ - elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", - snapshot); -} - - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * files reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeFiles(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->filearr)); -} - -/* - * Remember that a temporary file is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeFiles() - */ -void -ResourceOwnerRememberFile(ResourceOwner owner, File file) -{ - ResourceArrayAdd(&(owner->filearr), FileGetDatum(file)); -} - -/* - * Forget that a temporary file is owned by a ResourceOwner - */ -void -ResourceOwnerForgetFile(ResourceOwner owner, File file) -{ - if (!ResourceArrayRemove(&(owner->filearr), FileGetDatum(file))) - elog(ERROR, "temporary file %d is not owned by resource owner %s", - file, owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintFileLeakWarning(File file) -{ - elog(WARNING, "temporary file leak: File %d still referenced", - file); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * dynamic shmem segment reference array. - * - * This is separate from actually inserting an entry because if we run out - * of memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeDSMs(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->dsmarr)); -} - -/* - * Remember that a dynamic shmem segment is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeDSMs() - */ -void -ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg) -{ - ResourceArrayAdd(&(owner->dsmarr), PointerGetDatum(seg)); -} - -/* - * Forget that a dynamic shmem segment is owned by a ResourceOwner - */ -void -ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg) -{ - if (!ResourceArrayRemove(&(owner->dsmarr), PointerGetDatum(seg))) - elog(ERROR, "dynamic shared memory segment %u is not owned by resource owner %s", - dsm_segment_handle(seg), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintDSMLeakWarning(dsm_segment *seg) -{ - elog(WARNING, "dynamic shared memory leak: segment %u still referenced", - dsm_segment_handle(seg)); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * JIT context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeJIT(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->jitarr)); -} - -/* - * Remember that a JIT context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeJIT() - */ -void -ResourceOwnerRememberJIT(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->jitarr), handle); -} - -/* - * Forget that a JIT context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetJIT(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->jitarr), handle)) - elog(ERROR, "JIT context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Make sure there is room for at least one more entry in a ResourceOwner's - * cryptohash context reference array. - * - * This is separate from actually inserting an entry because if we run out of - * memory, it's critical to do so *before* acquiring the resource. - */ -void -ResourceOwnerEnlargeCryptoHash(ResourceOwner owner) -{ - ResourceArrayEnlarge(&(owner->cryptohasharr)); -} - -/* - * Remember that a cryptohash context is owned by a ResourceOwner - * - * Caller must have previously done ResourceOwnerEnlargeCryptoHash() - */ -void -ResourceOwnerRememberCryptoHash(ResourceOwner owner, Datum handle) -{ - ResourceArrayAdd(&(owner->cryptohasharr), handle); -} - -/* - * Forget that a cryptohash context is owned by a ResourceOwner - */ -void -ResourceOwnerForgetCryptoHash(ResourceOwner owner, Datum handle) -{ - if (!ResourceArrayRemove(&(owner->cryptohasharr), handle)) - elog(ERROR, "cryptohash context %p is not owned by resource owner %s", - DatumGetPointer(handle), owner->name); -} - -/* - * Debugging subroutine - */ -static void -PrintCryptoHashLeakWarning(Datum handle) -{ - elog(WARNING, "cryptohash context reference leak: context %p still referenced", - DatumGetPointer(handle)); -} diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 95704265b67..984af42e6da 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -66,7 +66,7 @@ #include "utils/memutils.h" #include "utils/old_snapshot.h" #include "utils/rel.h" -#include "utils/resowner_private.h" +#include "utils/resowner.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timestamp.h" @@ -174,6 +174,24 @@ static Snapshot CopySnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); +/* ResourceOwner callbacks to track snapshot references */ +static void ResOwnerReleaseSnapshot(Datum res); +static void ResOwnerPrintSnapshotLeakWarning(Datum res); + +static ResourceOwnerFuncs snapshot_resowner_funcs = +{ + .name = "snapshot reference", + .phase = RESOURCE_RELEASE_AFTER_LOCKS, + .ReleaseResource = ResOwnerReleaseSnapshot, + .PrintLeakWarning = ResOwnerPrintSnapshotLeakWarning +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberSnapshot(owner, snap) \ + ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) +#define ResourceOwnerForgetSnapshot(owner, snap) \ + ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_funcs) + /* * Snapshot fields to be serialized. * @@ -831,7 +849,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner) snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); /* and tell resowner.c about it */ - ResourceOwnerEnlargeSnapshots(owner); + ResourceOwnerEnlarge(owner); snap->regd_count++; ResourceOwnerRememberSnapshot(owner, snap); @@ -2349,3 +2367,19 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot) return false; } + +/* + * ResourceOwner callbacks + */ +static void +ResOwnerReleaseSnapshot(Datum res) +{ + UnregisterSnapshot((Snapshot) DatumGetPointer(res)); +} + +static void +ResOwnerPrintSnapshotLeakWarning(Datum res) +{ + elog(WARNING, "Snapshot reference leak: Snapshot %p still referenced", + DatumGetPointer(res)); +} diff --git a/src/common/cryptohash_openssl.c b/src/common/cryptohash_openssl.c index 643cc7aea2c..cf5fc9eacb4 100644 --- a/src/common/cryptohash_openssl.c +++ b/src/common/cryptohash_openssl.c @@ -30,7 +30,6 @@ #ifndef FRONTEND #include "utils/memutils.h" #include "utils/resowner.h" -#include "utils/resowner_private.h" #endif /* @@ -63,6 +62,27 @@ struct pg_cryptohash_ctx #endif }; +/* ResourceOwner callbacks to hold cryptohash contexts */ +#ifndef FRONTEND +static void ResOwnerReleaseCryptoHash(Datum res); +static void ResOwnerPrintCryptoHashLeakWarning(Datum res); + +static ResourceOwnerFuncs cryptohash_resowner_funcs = +{ + /* relcache references */ + .name = "OpenSSL cryptohash context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseCryptoHash, + .PrintLeakWarning = ResOwnerPrintCryptoHashLeakWarning, +}; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberCryptoHash(owner, ctx) \ + ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#define ResourceOwnerForgetCryptoHash(owner, ctx) \ + ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_funcs) +#endif + /* * pg_cryptohash_create * @@ -80,7 +100,7 @@ pg_cryptohash_create(pg_cryptohash_type type) * allocation to avoid leaking. */ #ifndef FRONTEND - ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); + ResourceOwnerEnlarge(CurrentResourceOwner); #endif ctx = ALLOC(sizeof(pg_cryptohash_ctx)); @@ -109,8 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type) #ifndef FRONTEND ctx->resowner = CurrentResourceOwner; - ResourceOwnerRememberCryptoHash(CurrentResourceOwner, - PointerGetDatum(ctx)); + ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx); #endif return ctx; @@ -241,10 +260,28 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx) EVP_MD_CTX_destroy(ctx->evpctx); #ifndef FRONTEND - ResourceOwnerForgetCryptoHash(ctx->resowner, - PointerGetDatum(ctx)); + ResourceOwnerForgetCryptoHash(ctx->resowner, ctx); #endif explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); FREE(ctx); } + + +/* + * ResourceOwner callbacks + */ +#ifndef FRONTEND +static void +ResOwnerReleaseCryptoHash(Datum res) +{ + pg_cryptohash_free((pg_cryptohash_ctx *) DatumGetPointer(res)); +} + +static void +ResOwnerPrintCryptoHashLeakWarning(Datum res) +{ + elog(WARNING, "cryptohash context reference leak: context %p still referenced", + DatumGetPointer(res)); +} +#endif diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h index f6b57829653..6815dc38a16 100644 --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -296,6 +296,15 @@ typedef struct CkptSortItem extern CkptSortItem *CkptBufferIds; +/* ResourceOwner callbacks to hold buffer pins */ +extern ResourceOwnerFuncs buffer_resowner_funcs; + +/* Convenience wrappers over ResourceOwnerRemember/Forget */ +#define ResourceOwnerRememberBuffer(owner, buffer) \ + ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) +#define ResourceOwnerForgetBuffer(owner, buffer) \ + ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_resowner_funcs) + /* * Internal buffer management routines */ diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index ddc2762eb3f..2bf95c22e84 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation, HeapTuple newtuple, void (*function) (int, uint32, Oid)); -extern void PrintCatCacheLeakWarning(HeapTuple tuple); -extern void PrintCatCacheListLeakWarning(CatCList *list); - #endif /* CATCACHE_H */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index ff09c63a02f..f3e3e50fb21 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -233,4 +233,6 @@ extern bool CachedPlanIsSimplyValid(CachedPlanSource *plansource, extern CachedExpression *GetCachedExpression(Node *expr); extern void FreeCachedExpression(CachedExpression *cexpr); +extern ResourceOwnerFuncs planref_resowner_funcs; + #endif /* PLANCACHE_H */ diff --git a/src/include/utils/resowner.h b/src/include/utils/resowner.h index 109ac31b248..d59d14c0d01 100644 --- a/src/include/utils/resowner.h +++ b/src/include/utils/resowner.h @@ -50,6 +50,36 @@ typedef enum RESOURCE_RELEASE_AFTER_LOCKS } ResourceReleasePhase; +/* + * In order to track an object, resowner.c needs a few callbacks for it. + * The callbacks for an resource of a specific kind are encapsulated in + * ResourceOwnerFuncs. + * + * Note that the callback occurs post-commit or post-abort, so these callback + * functions can only do noncritical cleanup. + */ +typedef struct ResourceOwnerFuncs +{ + const char *name; /* name for the object kind, for debugging */ + + ResourceReleasePhase phase; /* when are these objects released? */ + + /* + * Release resource. + * + * NOTE: this must call ResourceOwnerForget to disassociate it with the + * resource owner. + */ + void (*ReleaseResource)(Datum res); + + /* + * Print a warning, when a resource has not been properly released before + * commit. + */ + void (*PrintLeakWarning)(Datum res); + +} ResourceOwnerFuncs; + /* * Dynamically loaded modules can get control during ResourceOwnerRelease * by providing a callback of this form. @@ -71,16 +101,29 @@ extern void ResourceOwnerRelease(ResourceOwner owner, ResourceReleasePhase phase, bool isCommit, bool isTopLevel); -extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern void ResourceOwnerNewParent(ResourceOwner owner, ResourceOwner newparent); + +extern void ResourceOwnerEnlarge(ResourceOwner owner); +extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); +extern void ResourceOwnerForget(ResourceOwner owner, Datum res, ResourceOwnerFuncs *kind); + extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg); + extern void CreateAuxProcessResourceOwner(void); extern void ReleaseAuxProcessResources(bool isCommit); +/* special support for local lock management */ +struct LOCALLOCK; +extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock); +extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock); + +/* special function to relase all plancache references */ +extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner); + #endif /* RESOWNER_H */ diff --git a/src/include/utils/resowner_private.h b/src/include/utils/resowner_private.h deleted file mode 100644 index c480a1a24be..00000000000 --- a/src/include/utils/resowner_private.h +++ /dev/null @@ -1,105 +0,0 @@ -/*------------------------------------------------------------------------- - * - * resowner_private.h - * POSTGRES resource owner private definitions. - * - * See utils/resowner/README for more info. - * - * - * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * src/include/utils/resowner_private.h - * - *------------------------------------------------------------------------- - */ -#ifndef RESOWNER_PRIVATE_H -#define RESOWNER_PRIVATE_H - -#include "storage/dsm.h" -#include "storage/fd.h" -#include "storage/lock.h" -#include "utils/catcache.h" -#include "utils/plancache.h" -#include "utils/resowner.h" -#include "utils/snapshot.h" - - -/* support for buffer refcount management */ -extern void ResourceOwnerEnlargeBuffers(ResourceOwner owner); -extern void ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer); -extern void ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer); - -/* support for local lock management */ -extern void ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock); -extern void ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock); - -/* support for catcache refcount management */ -extern void ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerForgetCatCacheRef(ResourceOwner owner, - HeapTuple tuple); -extern void ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner); -extern void ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, - CatCList *list); -extern void ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, - CatCList *list); - -/* support for relcache refcount management */ -extern void ResourceOwnerEnlargeRelationRefs(ResourceOwner owner); -extern void ResourceOwnerRememberRelationRef(ResourceOwner owner, - Relation rel); -extern void ResourceOwnerForgetRelationRef(ResourceOwner owner, - Relation rel); - -/* support for plancache refcount management */ -extern void ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner); -extern void ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); -extern void ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, - CachedPlan *plan); - -/* support for tupledesc refcount management */ -extern void ResourceOwnerEnlargeTupleDescs(ResourceOwner owner); -extern void ResourceOwnerRememberTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); -extern void ResourceOwnerForgetTupleDesc(ResourceOwner owner, - TupleDesc tupdesc); - -/* support for snapshot refcount management */ -extern void ResourceOwnerEnlargeSnapshots(ResourceOwner owner); -extern void ResourceOwnerRememberSnapshot(ResourceOwner owner, - Snapshot snapshot); -extern void ResourceOwnerForgetSnapshot(ResourceOwner owner, - Snapshot snapshot); - -/* support for temporary file management */ -extern void ResourceOwnerEnlargeFiles(ResourceOwner owner); -extern void ResourceOwnerRememberFile(ResourceOwner owner, - File file); -extern void ResourceOwnerForgetFile(ResourceOwner owner, - File file); - -/* support for dynamic shared memory management */ -extern void ResourceOwnerEnlargeDSMs(ResourceOwner owner); -extern void ResourceOwnerRememberDSM(ResourceOwner owner, - dsm_segment *); -extern void ResourceOwnerForgetDSM(ResourceOwner owner, - dsm_segment *); - -/* support for JITContext management */ -extern void ResourceOwnerEnlargeJIT(ResourceOwner owner); -extern void ResourceOwnerRememberJIT(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetJIT(ResourceOwner owner, - Datum handle); - -/* support for cryptohash context management */ -extern void ResourceOwnerEnlargeCryptoHash(ResourceOwner owner); -extern void ResourceOwnerRememberCryptoHash(ResourceOwner owner, - Datum handle); -extern void ResourceOwnerForgetCryptoHash(ResourceOwner owner, - Datum handle); - -#endif /* RESOWNER_PRIVATE_H */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 8bd95aefa1d..7e511e02690 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2128,8 +2128,10 @@ ReplicationStateOnDisk ResTarget ReservoirState ReservoirStateData -ResourceArray +ResourceElem ResourceOwner +ResourceOwnerData +ResourceOwnerFuncs ResourceReleaseCallback ResourceReleaseCallbackItem ResourceReleasePhase -- 2.30.1 --------------C6A03312D0F99F9183A0DF46 Content-Type: text/x-patch; charset=UTF-8; name="v7-0003-Optimize-hash-function.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v7-0003-Optimize-hash-function.patch" ^ permalink raw reply [nested|flat] 113+ messages in thread
end of thread, other threads:[~2021-01-20 21:16 UTC | newest] Thread overview: 113+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2019-04-04 05:48 [PATCH v1] Fine tune documentation for tableam Justin Pryzby <[email protected]> 2019-04-04 05:48 [PATCH v1] Fine tune documentation for tableam Justin Pryzby <[email protected]> 2020-12-16 15:26 [PATCH v4 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v4 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v4 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v3 1/1] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v4 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v3 1/1] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v4 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v3 1/1] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v3 1/1] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v3 1/1] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v5 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v4 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v5 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v5 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v3 1/1] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v4 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v3 1/1] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v4 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v4 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v5 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v3 1/1] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v3 1/1] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v4 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v5 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v5 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v4 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v5 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v5 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v5 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v3 1/1] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v5 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v3 1/1] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v4 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v5 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v2 1/1] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v4 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v3 1/1] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v4 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v5 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v5 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v4 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v3 1/1] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v5 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v4 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v5 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v3 1/1] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v4 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v4 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v3 1/1] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v3 1/1] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v5 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v5 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v3 1/1] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v3 1/1] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v5 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v3 1/1] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v4 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v3 1/1] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v5 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v5 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v3 1/1] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v4 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v4 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2020-12-16 15:26 [PATCH v5 2/2] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v6 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v6 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v6 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v6 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v8 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v6 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v6 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v6 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v6 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v6 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v10 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v6 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v6 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v6 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v6 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v6 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v6 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v6 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v6 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v6 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v6 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v6 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v6 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]> 2021-01-20 21:16 [PATCH v7 2/3] Make resowners more easily extensible. Heikki Linnakangas <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox