agora inbox for [email protected]help / color / mirror / Atom feed
[PATCH 09/10] BRIN bloom indexes 14+ messages / 3 participants [nested] [flat]
* [PATCH 09/10] BRIN bloom indexes @ 2019-06-09 20:09 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 14+ messages in thread From: Tomas Vondra @ 2019-06-09 20:09 UTC (permalink / raw) --- doc/src/sgml/brin.sgml | 215 ++++++ doc/src/sgml/ref/create_index.sgml | 31 + src/backend/access/brin/Makefile | 2 +- src/backend/access/brin/brin.c | 22 +- src/backend/access/brin/brin_bloom.c | 858 +++++++++++++++++++++++ src/include/access/brin.h | 2 + src/include/access/brin_internal.h | 4 + src/include/catalog/pg_amop.dat | 165 +++++ src/include/catalog/pg_amproc.dat | 430 ++++++++++++ src/include/catalog/pg_opclass.dat | 69 ++ src/include/catalog/pg_opfamily.dat | 36 + src/include/catalog/pg_proc.dat | 20 + src/test/regress/expected/brin_bloom.out | 456 ++++++++++++ src/test/regress/expected/opr_sanity.out | 3 +- src/test/regress/parallel_schedule | 5 + src/test/regress/serial_schedule | 1 + src/test/regress/sql/brin_bloom.sql | 404 +++++++++++ 17 files changed, 2715 insertions(+), 8 deletions(-) create mode 100644 src/backend/access/brin/brin_bloom.c create mode 100644 src/test/regress/expected/brin_bloom.out create mode 100644 src/test/regress/sql/brin_bloom.sql diff --git a/doc/src/sgml/brin.sgml b/doc/src/sgml/brin.sgml index da0c911153..b73f0b7d2b 100644 --- a/doc/src/sgml/brin.sgml +++ b/doc/src/sgml/brin.sgml @@ -129,6 +129,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was </row> </thead> <tbody> + <row> + <entry><literal>int8_bloom_ops</literal></entry> + <entry><type>bigint</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>int8_minmax_ops</literal></entry> <entry><type>bigint</type></entry> @@ -180,6 +187,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <literal>|&></literal> </entry> </row> + <row> + <entry><literal>bytea_bloom_ops</literal></entry> + <entry><type>bytea</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>bytea_minmax_ops</literal></entry> <entry><type>bytea</type></entry> @@ -191,6 +205,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <literal>></literal> </entry> </row> + <row> + <entry><literal>bpchar_bloom_ops</literal></entry> + <entry><type>character</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>bpchar_minmax_ops</literal></entry> <entry><type>character</type></entry> @@ -202,6 +223,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <literal>></literal> </entry> </row> + <row> + <entry><literal>char_bloom_ops</literal></entry> + <entry><type>"char"</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>char_minmax_ops</literal></entry> <entry><type>"char"</type></entry> @@ -213,6 +241,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <literal>></literal> </entry> </row> + <row> + <entry><literal>date_bloom_ops</literal></entry> + <entry><type>date</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>date_minmax_ops</literal></entry> <entry><type>date</type></entry> @@ -224,6 +259,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <literal>></literal> </entry> </row> + <row> + <entry><literal>float8_bloom_ops</literal></entry> + <entry><type>double precision</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>float8_minmax_ops</literal></entry> <entry><type>double precision</type></entry> @@ -235,6 +277,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <literal>></literal> </entry> </row> + <row> + <entry><literal>inet_bloom_ops</literal></entry> + <entry><type>inet</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>inet_minmax_ops</literal></entry> <entry><type>inet</type></entry> @@ -258,6 +307,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <literal><<</literal> </entry> </row> + <row> + <entry><literal>int4_bloom_ops</literal></entry> + <entry><type>integer</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>int4_minmax_ops</literal></entry> <entry><type>integer</type></entry> @@ -269,6 +325,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <literal>></literal> </entry> </row> + <row> + <entry><literal>interval_bloom_ops</literal></entry> + <entry><type>interval</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>interval_minmax_ops</literal></entry> <entry><type>interval</type></entry> @@ -280,6 +343,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <literal>></literal> </entry> </row> + <row> + <entry><literal>macaddr_bloom_ops</literal></entry> + <entry><type>macaddr</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>macaddr_minmax_ops</literal></entry> <entry><type>macaddr</type></entry> @@ -291,6 +361,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <literal>></literal> </entry> </row> + <row> + <entry><literal>macaddr8_bloom_ops</literal></entry> + <entry><type>macaddr8</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>macaddr8_minmax_ops</literal></entry> <entry><type>macaddr8</type></entry> @@ -302,6 +379,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <literal>></literal> </entry> </row> + <row> + <entry><literal>name_bloom_ops</literal></entry> + <entry><type>name</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>name_minmax_ops</literal></entry> <entry><type>name</type></entry> @@ -313,6 +397,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <literal>></literal> </entry> </row> + <row> + <entry><literal>numeric_bloom_ops</literal></entry> + <entry><type>numeric</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>numeric_minmax_ops</literal></entry> <entry><type>numeric</type></entry> @@ -324,6 +415,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <literal>></literal> </entry> </row> + <row> + <entry><literal>pg_lsn_bloom_ops</literal></entry> + <entry><type>pg_lsn</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>pg_lsn_minmax_ops</literal></entry> <entry><type>pg_lsn</type></entry> @@ -335,6 +433,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <literal>></literal> </entry> </row> + <row> + <entry><literal>oid_bloom_ops</literal></entry> + <entry><type>oid</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>oid_minmax_ops</literal></entry> <entry><type>oid</type></entry> @@ -366,6 +471,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <literal>>=</literal> </entry> </row> + <row> + <entry><literal>float4_bloom_ops</literal></entry> + <entry><type>real</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>float4_minmax_ops</literal></entry> <entry><type>real</type></entry> @@ -377,6 +489,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <literal>></literal> </entry> </row> + <row> + <entry><literal>int2_bloom_ops</literal></entry> + <entry><type>smallint</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>int2_minmax_ops</literal></entry> <entry><type>smallint</type></entry> @@ -388,6 +507,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <literal>></literal> </entry> </row> + <row> + <entry><literal>text_bloom_ops</literal></entry> + <entry><type>text</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>text_minmax_ops</literal></entry> <entry><type>text</type></entry> @@ -410,6 +536,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <literal>></literal> </entry> </row> + <row> + <entry><literal>timestamp_bloom_ops</literal></entry> + <entry><type>timestamp without time zone</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>timestamp_minmax_ops</literal></entry> <entry><type>timestamp without time zone</type></entry> @@ -421,6 +554,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <literal>></literal> </entry> </row> + <row> + <entry><literal>timestamptz_bloom_ops</literal></entry> + <entry><type>timestamp with time zone</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>timestamptz_minmax_ops</literal></entry> <entry><type>timestamp with time zone</type></entry> @@ -432,6 +572,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <literal>></literal> </entry> </row> + <row> + <entry><literal>time_bloom_ops</literal></entry> + <entry><type>time without time zone</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>time_minmax_ops</literal></entry> <entry><type>time without time zone</type></entry> @@ -443,6 +590,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <literal>></literal> </entry> </row> + <row> + <entry><literal>timetz_bloom_ops</literal></entry> + <entry><type>time with time zone</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>timetz_minmax_ops</literal></entry> <entry><type>time with time zone</type></entry> @@ -454,6 +608,13 @@ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was <literal>></literal> </entry> </row> + <row> + <entry><literal>uuid_bloom_ops</literal></entry> + <entry><type>uuid</type></entry> + <entry> + <literal>=</literal> + </entry> + </row> <row> <entry><literal>uuid_minmax_ops</literal></entry> <entry><type>uuid</type></entry> @@ -803,6 +964,60 @@ typedef struct BrinOpcInfo function can improve index performance. </para> + <para> + To write an operator class for a data type that implements only an equality + operator and supports hashing, it is possible to use the bloom support procedures + alongside the corresponding operators, as shown in + <xref linkend="brin-extensibility-bloom-table"/>. + All operator class members (procedures and operators) are mandatory. + </para> + + <table id="brin-extensibility-bloom-table"> + <title>Procedure and Support Numbers for Bloom Operator Classes</title> + <tgroup cols="2"> + <thead> + <row> + <entry>Operator class member</entry> + <entry>Object</entry> + </row> + </thead> + <tbody> + <row> + <entry>Support Procedure 1</entry> + <entry>internal function <function>brin_bloom_opcinfo()</function></entry> + </row> + <row> + <entry>Support Procedure 2</entry> + <entry>internal function <function>brin_bloom_add_value()</function></entry> + </row> + <row> + <entry>Support Procedure 3</entry> + <entry>internal function <function>brin_bloom_consistent()</function></entry> + </row> + <row> + <entry>Support Procedure 4</entry> + <entry>internal function <function>brin_bloom_union()</function></entry> + </row> + <row> + <entry>Support Procedure 11</entry> + <entry>function to compute hash of an element</entry> + </row> + <row> + <entry>Operator Strategy 1</entry> + <entry>operator equal-to</entry> + </row> + </tbody> + </tgroup> + </table> + + <para> + Support procedure numbers 1-10 are reserved for the BRIN internal + functions, so the SQL level functions start with number 11. Support + function number 11 is the main function required to build the index. + It should accept one argument with the same data type as the operator class, + and return a hash of the value. + </para> + <para> Both minmax and inclusion operator classes support cross-data-type operators, though with these the dependencies become more complicated. diff --git a/doc/src/sgml/ref/create_index.sgml b/doc/src/sgml/ref/create_index.sgml index 61401f3645..05b2b01614 100644 --- a/doc/src/sgml/ref/create_index.sgml +++ b/doc/src/sgml/ref/create_index.sgml @@ -523,6 +523,37 @@ CREATE [ UNIQUE ] INDEX [ CONCURRENTLY ] [ [ IF NOT EXISTS ] <replaceable class= </para> </listitem> </varlistentry> + + <varlistentry> + <term><literal>n_distinct_per_range</literal></term> + <listitem> + <para> + Defines the estimated number of distinct non-null values in the block + range, used by <acronym>BRIN</acronym> bloom indexes for sizing of the + Bloom filter. It behaves similarly to <literal>n_distinct</literal> option + for <xref linkend="sql-altertable"/>. When set to a positive value, + each block range is assumed to contain this number of distinct non-null + values. When set to a negative value, which must be greater than or + equal to -1, the number of distinct non-null is assumed linear with + the maximum possible number of tuples in the block range (about 290 + rows per block). The default values is <literal>-0.1</literal>, and + the minimum number of distinct non-null values is <literal>128</literal>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>false_positive_rate</literal></term> + <listitem> + <para> + Defines the desired false positive rate used by <acronym>BRIN</acronym> + bloom indexes for sizing of the Bloom filter. The values must be be + greater than 0.0 and smaller than 1.0. The default values is 0.01, + which is 1% false positive rate. + </para> + </listitem> + </varlistentry> + </variablelist> </refsect2> diff --git a/src/backend/access/brin/Makefile b/src/backend/access/brin/Makefile index 5aef925ed4..a76d927ee0 100644 --- a/src/backend/access/brin/Makefile +++ b/src/backend/access/brin/Makefile @@ -13,6 +13,6 @@ top_builddir = ../../../.. include $(top_builddir)/src/Makefile.global OBJS = brin.o brin_pageops.o brin_revmap.o brin_tuple.o brin_xlog.o \ - brin_minmax.o brin_inclusion.o brin_validate.o + brin_minmax.o brin_inclusion.o brin_validate.o brin_bloom.o include $(top_srcdir)/src/backend/common.mk diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c index 67f42ed5d1..8878bf10f7 100644 --- a/src/backend/access/brin/brin.c +++ b/src/backend/access/brin/brin.c @@ -53,6 +53,7 @@ typedef struct BrinBuildState BrinRevmap *bs_rmAccess; BrinDesc *bs_bdesc; BrinMemTuple *bs_dtuple; + bytea **opclassOptions; } BrinBuildState; /* @@ -77,7 +78,8 @@ static void union_tuples(BrinDesc *bdesc, BrinMemTuple *a, BrinTuple *b); static void brin_vacuum_scan(Relation idxrel, BufferAccessStrategy strategy); static bool add_values_to_range(Relation idxRel, BrinDesc *bdesc, - BrinMemTuple *dtup, Datum *values, bool *nulls); + BrinMemTuple *dtup, Datum *values, bool *nulls, + bytea **options); static bool check_null_keys(BrinValues *bval, ScanKey *nullkeys, int nnullkeys); /* @@ -238,7 +240,8 @@ brininsert(Relation idxRel, Datum *values, bool *nulls, dtup = brin_deform_tuple(bdesc, brtup, NULL); - need_insert = add_values_to_range(idxRel, bdesc, dtup, values, nulls); + /* FIXME use correct opclass options */ + need_insert = add_values_to_range(idxRel, bdesc, dtup, values, nulls, NULL); if (!need_insert) { @@ -736,7 +739,7 @@ brinbuildCallback(Relation index, /* Accumulate the current tuple into the running state */ (void) add_values_to_range(index, state->bs_bdesc, state->bs_dtuple, - values, isnull); + values, isnull, state->opclassOptions); } /* @@ -1218,6 +1221,7 @@ initialize_brin_buildstate(Relation idxRel, BrinRevmap *revmap, state->bs_rmAccess = revmap; state->bs_bdesc = brin_build_desc(idxRel); state->bs_dtuple = brin_new_memtuple(state->bs_bdesc); + state->opclassOptions = RelationGetParsedOpclassOptions(idxRel); brin_memtuple_initialize(state->bs_dtuple, state->bs_bdesc); @@ -1250,6 +1254,7 @@ terminate_brin_buildstate(BrinBuildState *state) brin_free_desc(state->bs_bdesc); pfree(state->bs_dtuple); + pfree(state->opclassOptions); pfree(state); } @@ -1626,7 +1631,7 @@ brin_vacuum_scan(Relation idxrel, BufferAccessStrategy strategy) static bool add_values_to_range(Relation idxRel, BrinDesc *bdesc, BrinMemTuple *dtup, - Datum *values, bool *nulls) + Datum *values, bool *nulls, bytea **options) { int keyno; bool modified = false; @@ -1643,6 +1648,7 @@ add_values_to_range(Relation idxRel, BrinDesc *bdesc, BrinMemTuple *dtup, Datum result; BrinValues *bval; FmgrInfo *addValue; + bytea *opts; bval = &dtup->bt_columns[keyno]; @@ -1661,14 +1667,18 @@ add_values_to_range(Relation idxRel, BrinDesc *bdesc, BrinMemTuple *dtup, continue; } + opts = (options) ? options[keyno] : NULL; + + addValue = index_getprocinfo(idxRel, keyno + 1, BRIN_PROCNUM_ADDVALUE); - result = FunctionCall4Coll(addValue, + result = FunctionCall5Coll(addValue, idxRel->rd_indcollation[keyno], PointerGetDatum(bdesc), PointerGetDatum(bval), values[keyno], - nulls[keyno]); + nulls[keyno], + PointerGetDatum(opts)); /* if that returned true, we need to insert the updated tuple */ modified |= DatumGetBool(result); } diff --git a/src/backend/access/brin/brin_bloom.c b/src/backend/access/brin/brin_bloom.c new file mode 100644 index 0000000000..ef026a72d5 --- /dev/null +++ b/src/backend/access/brin/brin_bloom.c @@ -0,0 +1,858 @@ +/* + * brin_bloom.c + * Implementation of Bloom opclass for BRIN + * + * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * IDENTIFICATION + * src/backend/access/brin/brin_bloom.c + */ +#include "postgres.h" + +#include "access/genam.h" +#include "access/brin.h" +#include "access/brin_internal.h" +#include "access/brin_tuple.h" +#include "access/hash.h" +#include "access/htup_details.h" +#include "access/reloptions.h" +#include "access/stratnum.h" +#include "catalog/pg_type.h" +#include "catalog/pg_amop.h" +#include "utils/builtins.h" +#include "utils/datum.h" +#include "utils/lsyscache.h" +#include "utils/rel.h" +#include "utils/syscache.h" + +#include <math.h> + +#define BloomEqualStrategyNumber 1 + +/* + * Additional SQL level support functions + * + * Procedure numbers must not use values reserved for BRIN itself; see + * brin_internal.h. + */ +#define BLOOM_MAX_PROCNUMS 4 /* maximum support procs we need */ +#define PROCNUM_HASH 11 /* required */ + +/* + * Subtract this from procnum to obtain index in BloomOpaque arrays + * (Must be equal to minimum of private procnums). + */ +#define PROCNUM_BASE 11 + + +#define BLOOM_PHASE_SORTED 1 +#define BLOOM_PHASE_HASH 2 + +/* how many hashes to accumulate before hashing */ +#define BLOOM_MAX_UNSORTED 32 +#define BLOOM_GROW_BYTES 32 + +/* + * Storage type for BRIN's reloptions + */ +typedef struct BloomOptions +{ + int32 vl_len_; /* varlena header (do not touch directly!) */ + double nDistinctPerRange; /* number of distinct values per range */ + double falsePositiveRate; /* false positive for bloom filter */ +} BloomOptions; + +/* default values for bloom-specific reloptions */ +#define BLOOM_MIN_NDISTINCT_PER_RANGE 128 +#define BLOOM_DEFAULT_NDISTINCT_PER_RANGE -0.1 /* 10% of values */ +#define BLOOM_DEFAULT_FALSE_POSITIVE_RATE 0.01 /* 1% fp rate */ + +#define BloomGetNDistinctPerRange(opts) \ + ((opts) && (((BloomOptions *) (opts))->nDistinctPerRange != 0) ? \ + (((BloomOptions *) (opts))->nDistinctPerRange) : \ + BLOOM_DEFAULT_NDISTINCT_PER_RANGE) + +#define BloomGetFalsePositiveRate(opts) \ + ((opts) && (((BloomOptions *) (opts))->falsePositiveRate != 0.0) ? \ + (((BloomOptions *) (opts))->falsePositiveRate) : \ + BLOOM_DEFAULT_FALSE_POSITIVE_RATE) + + +/* + * Bloom Filter + * + * Represents a bloom filter, built on hashes of the indexed values. That is, + * we compute a uint32 hash of the value, and then store this hash into the + * bloom filter (and compute additional hashes on it). + * + * We use an optimisation that initially we store the uint32 values directly, + * without the extra hashing step. And only later filling the bitmap space, + * we switch to the regular bloom filter mode. + * + * PHASE_SORTED + * + * Initially we copy the uint32 hash into the bitmap, regularly sorting the + * hash values for fast lookup (we keep at most BLOOM_MAX_UNSORTED unsorted + * values). + * + * The idea is that if we only see very few distinct values, we can store + * them in less space compared to the (sparse) bloom filter bitmap. It also + * stores them exactly, although that's not a big advantage as almost-empty + * bloom filter has false positive rate close to zero anyway. + * + * PHASE_HASH + * + * Once we fill the bitmap space in the sorted phase, we switch to the hash + * phase, where we actually use the bloom filter. We treat the uint32 hashes + * as input values, and hash them again with different seeds (to get the k + * hash functions needed for bloom filter). + * + * + * XXX Maybe instead of explicitly limiting the number of unsorted values + * by BLOOM_MAX_UNSORTED, we should cap them by (filter size / 4B), i.e. + * allow up to the whole filter size. + * + * XXX Perhaps we could save a few bytes by using different data types, but + * considering the size of the bitmap, the difference is negligible. + * + * XXX We could also implement "sparse" bloom filters, keeping only the + * bytes that are not entirely 0. That might make the "sorted" phase + * mostly unnecessary. + * + * XXX We can also watch the number of bits set in the bloom filter, and + * then stop using it (and not store the bitmap, to save space) when the + * false positive rate gets too high. + */ +typedef struct BloomFilter +{ + /* varlena header (do not touch directly!) */ + int32 vl_len_; + + /* global bloom filter parameters */ + uint32 phase; /* phase (initially SORTED, then HASH) */ + uint32 nhashes; /* number of hash functions */ + uint32 nbits; /* number of bits in the bitmap (optimal) */ + uint32 nbits_set; /* number of bits set to 1 */ + + /* fields used only in the EXACT phase */ + uint32 nvalues; /* number of hashes stored (sorted + extra) */ + uint32 nsorted; /* number of uint32 hashes in sorted part */ + + /* bitmap of the bloom filter */ + char bitmap[FLEXIBLE_ARRAY_MEMBER]; + +} BloomFilter; + +/* + * bloom_init + * Initialize the Bloom Filter, allocate all the memory. + * + * The filter is initialized with optimal size for ndistinct expected + * values, and requested false positive rate. The filter is stored as + * varlena. + */ +static BloomFilter * +bloom_init(int ndistinct, double false_positive_rate) +{ + Size len; + BloomFilter *filter; + + /* https://en.wikipedia.org/wiki/Bloom_filter */ + int m; /* number of bits */ + double k; /* number of hash functions */ + + Assert(ndistinct > 0); + Assert((false_positive_rate > 0) && (false_positive_rate < 1.0)); + + m = ceil((ndistinct * log(false_positive_rate)) / log(1.0 / (pow(2.0, log(2.0))))); + + /* round m to whole bytes */ + m = ((m + 7) / 8) * 8; + + /* + * round(log(2.0) * m / ndistinct), but assume round() may not be + * available on Windows + */ + k = log(2.0) * m / ndistinct; + k = (k - floor(k) >= 0.5) ? ceil(k) : floor(k); + + /* + * Allocate the bloom filter with a minimum size 64B (about 40B in the + * bitmap part). We require space at least for the header. + */ + len = Max(offsetof(BloomFilter, bitmap), 64); + + filter = (BloomFilter *) palloc0(len); + + filter->phase = BLOOM_PHASE_SORTED; + filter->nhashes = (int) k; + filter->nbits = m; + + SET_VARSIZE(filter, len); + + return filter; +} + +/* simple uint32 comparator, for pg_qsort and bsearch */ +static int +cmp_uint32(const void *a, const void *b) +{ + uint32 *ia = (uint32 *) a; + uint32 *ib = (uint32 *) b; + + if (*ia == *ib) + return 0; + else if (*ia < *ib) + return -1; + else + return 1; +} + +/* + * bloom_compact + * Compact the filter during the 'sorted' phase. + * + * We sort the uint32 hashes and remove duplicates, for two main reasons. + * Firstly, to keep most of the data sorted for bsearch lookups. Secondly, + * we try to save space by removing the duplicates, allowing us to stay + * in the sorted phase a bit longer. + * + * We currently don't repalloc the bitmap, i.e. we don't free the memory + * here - in the worst case we waste space for up to 32 unsorted hashes + * (if all of them are already in the sorted part), so about 128B. We can + * either reduce the number of unsorted items (e.g. to 8 hashes, which + * would mean 32B), or start doing the repalloc. + * + * XXX Actually, we don't need to do repalloc - we just need to set the + * varlena header length! + */ +static void +bloom_compact(BloomFilter *filter) +{ + int i, + nvalues; + uint32 *values; + + /* never call compact on filters in HASH phase */ + Assert(filter->phase == BLOOM_PHASE_SORTED); + + /* no chance to compact anything */ + if (filter->nvalues == filter->nsorted) + return; + + values = (uint32 *) palloc(filter->nvalues * sizeof(uint32)); + + /* copy the data, then reset the bitmap */ + memcpy(values, filter->bitmap, filter->nvalues * sizeof(uint32)); + memset(filter->bitmap, 0, filter->nvalues * sizeof(uint32)); + + /* FIXME optimization: sort only the unsorted part, then merge */ + pg_qsort(values, filter->nvalues, sizeof(uint32), cmp_uint32); + + nvalues = 1; + for (i = 1; i < filter->nvalues; i++) + { + /* if a new value, keep it */ + if (values[i] != values[i-1]) + { + values[nvalues] = values[i]; + nvalues++; + } + } + + filter->nvalues = nvalues; + filter->nsorted = nvalues; + + memcpy(filter->bitmap, values, nvalues * sizeof(uint32)); + + pfree(values); +} + +static BloomFilter * +bloom_switch_to_hashing(BloomFilter *filter); + +/* + * bloom_add_value + * Add value to the bloom filter. + */ +static BloomFilter * +bloom_add_value(BloomFilter *filter, uint32 value, bool *updated) +{ + int i; + uint32 big_h, h, d; + + /* assume 'not updated' by default */ + Assert(filter); + + /* if we're in the sorted phase, we store the hashes directly */ + if (filter->phase == BLOOM_PHASE_SORTED) + { + /* how many uint32 hashes can we fit into the bitmap */ + int maxvalues = filter->nbits / (8 * sizeof(uint32)); + + /* do not overflow the bitmap space or number of unsorted items */ + Assert(filter->nvalues <= maxvalues); + Assert(filter->nvalues - filter->nsorted <= BLOOM_MAX_UNSORTED); + + /* + * In this branch we always update the filter - we either add the + * hash to the unsorted part, or switch the filter to hashing. + */ + if (updated) + *updated = true; + + /* + * If the array is full, or if we reached the limit on unsorted + * items, try to compact the filter first, before attempting to + * add the new value. + */ + if ((filter->nvalues == maxvalues) || + (filter->nvalues - filter->nsorted == BLOOM_MAX_UNSORTED)) + bloom_compact(filter); + + /* + * Can we squeeze one more uint32 hash into the bitmap? Also make + * sure there's enough space in the bytea value first. + */ + if (filter->nvalues < maxvalues) + { + Size len = VARSIZE_ANY(filter); + Size need = offsetof(BloomFilter,bitmap) + (filter->nvalues+1) * sizeof(uint32); + + if (len < need) + { + /* + * We don't double the size here, as in the first place we care about + * reducing storage requirements, and the doubling happens automatically + * in memory contexts anyway. + * + * XXX Zero the newly allocated part. Maybe not really needed? + */ + filter = (BloomFilter *) repalloc(filter, len + BLOOM_GROW_BYTES); + memset((char *)filter + len, 0, BLOOM_GROW_BYTES); + SET_VARSIZE(filter, len + BLOOM_GROW_BYTES); + } + + /* copy the data into the bitmap */ + memcpy(&filter->bitmap[filter->nvalues * sizeof(uint32)], + &value, sizeof(uint32)); + + filter->nvalues++; + + /* we're done */ + return filter; + } + + /* can't add any more exact hashes, so switch to hashing */ + filter = bloom_switch_to_hashing(filter); + } + + /* we better be in the hashing phase */ + Assert(filter->phase == BLOOM_PHASE_HASH); + + /* compute the hashes, used for the bloom filter */ + big_h = ((uint32)DatumGetInt64(hash_uint32(value))); + + h = big_h % filter->nbits; + d = big_h % (filter->nbits - 1); + + /* compute the requested number of hashes */ + for (i = 0; i < filter->nhashes; i++) + { + int byte = (h / 8); + int bit = (h % 8); + + /* if the bit is not set, set it and remember we did that */ + if (! (filter->bitmap[byte] & (0x01 << bit))) + { + filter->bitmap[byte] |= (0x01 << bit); + filter->nbits_set++; + if (updated) + *updated = true; + } + + /* next bit */ + h += d++; + if (h >= filter->nbits) + h -= filter->nbits; + + if (d == filter->nbits) + d = 0; + } + + return filter; +} + +/* + * bloom_switch_to_hashing + * Switch the bloom filter from sorted to hashing mode. + */ +static BloomFilter * +bloom_switch_to_hashing(BloomFilter *filter) +{ + int i; + uint32 *values; + Size len; + BloomFilter *newfilter; + + /* + * The new filter is allocated with all the memory, directly into + * the HASH phase. + */ + len = offsetof(BloomFilter, bitmap) + (filter->nbits / 8); + + newfilter = (BloomFilter *) palloc0(len); + + newfilter->nhashes = filter->nhashes; + newfilter->nbits = filter->nbits; + newfilter->phase = BLOOM_PHASE_HASH; + + SET_VARSIZE(newfilter, len); + + values = (uint32 *) filter->bitmap; + + for (i = 0; i < filter->nvalues; i++) + /* ignore the return value here, re don't repalloc in hashing mode */ + bloom_add_value(newfilter, values[i], NULL); + + /* free the original filter, return the newly allocated one */ + pfree(filter); + + return newfilter; +} + +/* + * bloom_contains_value + * Check if the bloom filter contains a particular value. + */ +static bool +bloom_contains_value(BloomFilter *filter, uint32 value) +{ + int i; + uint32 big_h, h, d; + + Assert(filter); + + /* in sorted mode we simply search the two arrays (sorted, unsorted) */ + if (filter->phase == BLOOM_PHASE_SORTED) + { + int i; + uint32 *values = (uint32 *)filter->bitmap; + + /* first search through the sorted part */ + if ((filter->nsorted > 0) && + (bsearch(&value, values, filter->nsorted, sizeof(uint32), cmp_uint32) != NULL)) + return true; + + /* now search through the unsorted part - linear search */ + for (i = filter->nsorted; i < filter->nvalues; i++) + if (value == values[i]) + return true; + + /* nothing found */ + return false; + } + + /* now the regular hashing mode */ + Assert(filter->phase == BLOOM_PHASE_HASH); + + big_h = ((uint32)DatumGetInt64(hash_uint32(value))); + + h = big_h % filter->nbits; + d = big_h % (filter->nbits - 1); + + /* compute the requested number of hashes */ + for (i = 0; i < filter->nhashes; i++) + { + int byte = (h / 8); + int bit = (h % 8); + + /* if the bit is not set, the value is not there */ + if (! (filter->bitmap[byte] & (0x01 << bit))) + return false; + + /* next bit */ + h += d++; + if (h >= filter->nbits) + h -= filter->nbits; + + if (d == filter->nbits) + d = 0; + } + + /* all hashes found in bloom filter */ + return true; +} + +typedef struct BloomOpaque +{ + /* + * XXX At this point we only need a single proc (to compute the hash), + * but let's keep the array just like inclusion and minman opclasses, + * for consistency. We may need additional procs in the future. + */ + FmgrInfo extra_procinfos[BLOOM_MAX_PROCNUMS]; + bool extra_proc_missing[BLOOM_MAX_PROCNUMS]; +} BloomOpaque; + +static FmgrInfo *bloom_get_procinfo(BrinDesc *bdesc, uint16 attno, + uint16 procnum); + + +Datum +brin_bloom_opcinfo(PG_FUNCTION_ARGS) +{ + BrinOpcInfo *result; + + /* + * opaque->strategy_procinfos is initialized lazily; here it is set to + * all-uninitialized by palloc0 which sets fn_oid to InvalidOid. + * + * bloom indexes only store a the filter as a single BYTEA column + */ + + result = palloc0(MAXALIGN(SizeofBrinOpcInfo(1)) + + sizeof(BloomOpaque)); + result->oi_nstored = 1; + result->oi_regular_nulls = true; + result->oi_opaque = (BloomOpaque *) + MAXALIGN((char *) result + SizeofBrinOpcInfo(1)); + result->oi_typcache[0] = lookup_type_cache(BYTEAOID, 0); + + PG_RETURN_POINTER(result); +} + +/* + * brin_bloom_get_ndistinct + * Determine the ndistinct value used to size bloom filter. + * + * Tweak the ndistinct value based on the pagesPerRange value. First, + * if it's negative, it's assumed to be relative to maximum number of + * tuples in the range (assuming each page gets MaxHeapTuplesPerPage + * tuples, which is likely a significant over-estimate). We also clamp + * the value, not to over-size the bloom filter unnecessarily. + * + * XXX We can only do this when the pagesPerRange value was supplied. + * If it wasn't, it has to be a read-only access to the index, in which + * case we don't really care. But perhaps we should fall-back to the + * default pagesPerRange value? + * + * XXX We might also fetch info about ndistinct for the column(s) and + * compute the expected number of distinct values in a range. But that + * may be tricky due to data being sorted in various ways, so it seems + * better to rely on the upper estimate. + */ +static int +brin_bloom_get_ndistinct(BrinDesc *bdesc, BloomOptions *opts) +{ + double ndistinct; + double maxtuples; + BlockNumber pagesPerRange; + + pagesPerRange = BrinGetPagesPerRange(bdesc->bd_index); + ndistinct = BloomGetNDistinctPerRange(opts); + + Assert(BlockNumberIsValid(pagesPerRange)); + + maxtuples = MaxHeapTuplesPerPage * pagesPerRange; + + /* + * Similarly to n_distinct, negative values are relative - in this + * case to maximum number of tuples in the page range (maxtuples). + */ + if (ndistinct < 0) + ndistinct = (-ndistinct) * maxtuples; + + /* + * Positive values are to be used directly, but we still apply a + * couple of safeties no to use unnecessarily small bloom filters. + * We never use less than 128, because we would not save much space + * anyway. MaxHeapTuplesPerPage is ~290, so this is still lower. + */ + ndistinct = Max(ndistinct, BLOOM_MIN_NDISTINCT_PER_RANGE); + + /* + * And don't use more than the maximum possible number of tuples, + * in the range, which would be entirely wasteful. + */ + ndistinct = Min(ndistinct, maxtuples); + + return (int) ndistinct; +} + +static double +brin_bloom_get_fp_rate(BrinDesc *bdesc, BloomOptions *opts) +{ + return BloomGetFalsePositiveRate(opts); +} + +/* + * Examine the given index tuple (which contains partial status of a certain + * page range) by comparing it to the given value that comes from another heap + * tuple. If the new value is outside the bloom filter specified by the + * existing tuple values, update the index tuple and return true. Otherwise, + * return false and do not modify in this case. + */ +Datum +brin_bloom_add_value(PG_FUNCTION_ARGS) +{ + BrinDesc *bdesc = (BrinDesc *) PG_GETARG_POINTER(0); + BrinValues *column = (BrinValues *) PG_GETARG_POINTER(1); + Datum newval = PG_GETARG_DATUM(2); + bool isnull PG_USED_FOR_ASSERTS_ONLY = PG_GETARG_DATUM(3); + BloomOptions *opts = (BloomOptions *) PG_GETARG_POINTER(4); + Oid colloid = PG_GET_COLLATION(); + FmgrInfo *hashFn; + uint32 hashValue; + bool updated = false; + AttrNumber attno; + BloomFilter *filter; + + Assert(!isnull); + + attno = column->bv_attno; + + /* + * If this is the first non-null value, we need to initialize the bloom + * filter. Otherwise just extract the existing bloom filter from BrinValues. + */ + if (column->bv_allnulls) + { + filter = bloom_init(brin_bloom_get_ndistinct(bdesc, opts), + brin_bloom_get_fp_rate(bdesc, opts)); + column->bv_values[0] = PointerGetDatum(filter); + column->bv_allnulls = false; + updated = true; + } + else + filter = (BloomFilter *) PG_DETOAST_DATUM(column->bv_values[0]); + + /* + * Compute the hash of the new value, using the supplied hash function, + * and then add the hash value to the bloom filter. + */ + hashFn = bloom_get_procinfo(bdesc, attno, PROCNUM_HASH); + + hashValue = DatumGetUInt32(FunctionCall1Coll(hashFn, colloid, newval)); + + filter = bloom_add_value(filter, hashValue, &updated); + + column->bv_values[0] = PointerGetDatum(filter); + + PG_RETURN_BOOL(updated); +} + +/* + * Given an index tuple corresponding to a certain page range and a scan key, + * return whether the scan key is consistent with the index tuple's bloom + * filter. Return true if so, false otherwise. + */ +Datum +brin_bloom_consistent(PG_FUNCTION_ARGS) +{ + BrinDesc *bdesc = (BrinDesc *) PG_GETARG_POINTER(0); + BrinValues *column = (BrinValues *) PG_GETARG_POINTER(1); + ScanKey *keys = (ScanKey *) PG_GETARG_POINTER(2); + int nkeys = PG_GETARG_INT32(3); + // BloomOptions *opts = (BloomOptions *) PG_GETARG_POINTER(4); + Oid colloid = PG_GET_COLLATION(); + AttrNumber attno; + Datum value; + Datum matches; + FmgrInfo *finfo; + uint32 hashValue; + BloomFilter *filter; + int keyno; + + filter = (BloomFilter *) PG_DETOAST_DATUM(column->bv_values[0]); + + Assert(filter); + + matches = true; + + for (keyno = 0; keyno < nkeys; keyno++) + { + ScanKey key = keys[keyno]; + + /* NULL keys are handled and filtered-out in bringetbitmap */ + Assert(!(key->sk_flags & SK_ISNULL)); + + attno = key->sk_attno; + value = key->sk_argument; + + switch (key->sk_strategy) + { + case BloomEqualStrategyNumber: + + /* + * In the equality case (WHERE col = someval), we want to return + * the current page range if the minimum value in the range <= + * scan key, and the maximum value >= scan key. + */ + finfo = bloom_get_procinfo(bdesc, attno, PROCNUM_HASH); + + hashValue = DatumGetUInt32(FunctionCall1Coll(finfo, colloid, value)); + matches &= bloom_contains_value(filter, hashValue); + + break; + default: + /* shouldn't happen */ + elog(ERROR, "invalid strategy number %d", key->sk_strategy); + matches = 0; + break; + } + + if (!matches) + break; + } + + PG_RETURN_DATUM(matches); +} + +/* + * Given two BrinValues, update the first of them as a union of the summary + * values contained in both. The second one is untouched. + * + * XXX We assume the bloom filters have the same parameters fow now. In the + * future we should have 'can union' function, to decide if we can combine + * two particular bloom filters. + */ +Datum +brin_bloom_union(PG_FUNCTION_ARGS) +{ + BrinValues *col_a = (BrinValues *) PG_GETARG_POINTER(1); + BrinValues *col_b = (BrinValues *) PG_GETARG_POINTER(2); + // BloomOptions *opts = (BloomOptions *) PG_GETARG_POINTER(3); + BloomFilter *filter_a; + BloomFilter *filter_b; + + Assert(col_a->bv_attno == col_b->bv_attno); + Assert(!col_a->bv_allnulls && !col_b->bv_allnulls); + + filter_a = (BloomFilter *) PG_DETOAST_DATUM(col_a->bv_values[0]); + filter_b = (BloomFilter *) PG_DETOAST_DATUM(col_b->bv_values[0]); + + /* make sure neither of the bloom filters is NULL */ + Assert(filter_a && filter_b); + Assert(filter_a->nbits == filter_b->nbits); + + /* + * Merging of the filters depends on the phase of both filters. + */ + if (filter_b->phase == BLOOM_PHASE_SORTED) + { + /* + * Simply read all items from 'b' and add them to 'a' (the phase of + * 'a' does not really matter). + */ + int i; + uint32 *values = (uint32 *) filter_b->bitmap; + + for (i = 0; i < filter_b->nvalues; i++) + filter_a = bloom_add_value(filter_a, values[i], NULL); + + col_a->bv_values[0] = PointerGetDatum(filter_a); + } + else if (filter_a->phase == BLOOM_PHASE_SORTED) + { + /* + * 'b' hashed, 'a' sorted - copy 'b' into 'a' and then add all values + * from 'a' into the new copy. + */ + int i; + BloomFilter *filter_c; + uint32 *values = (uint32 *) filter_a->bitmap; + + filter_c = (BloomFilter *) PG_DETOAST_DATUM(datumCopy(PointerGetDatum(filter_b), false, -1)); + + for (i = 0; i < filter_a->nvalues; i++) + filter_c = bloom_add_value(filter_c, values[i], NULL); + + col_a->bv_values[0] = PointerGetDatum(filter_c); + } + else if (filter_a->phase == BLOOM_PHASE_HASH) + { + /* + * 'b' hashed, 'a' hashed - merge the bitmaps by OR + */ + int i; + int nbytes = (filter_a->nbits + 7) / 8; + + for (i = 0; i < nbytes; i++) + filter_a->bitmap[i] |= filter_b->bitmap[i]; + } + + PG_RETURN_VOID(); +} + +/* + * Cache and return inclusion opclass support procedure + * + * Return the procedure corresponding to the given function support number + * or null if it is not exists. + */ +static FmgrInfo * +bloom_get_procinfo(BrinDesc *bdesc, uint16 attno, uint16 procnum) +{ + BloomOpaque *opaque; + uint16 basenum = procnum - PROCNUM_BASE; + + /* + * We cache these in the opaque struct, to avoid repetitive syscache + * lookups. + */ + opaque = (BloomOpaque *) bdesc->bd_info[attno - 1]->oi_opaque; + + /* + * If we already searched for this proc and didn't find it, don't bother + * searching again. + */ + if (opaque->extra_proc_missing[basenum]) + return NULL; + + if (opaque->extra_procinfos[basenum].fn_oid == InvalidOid) + { + if (RegProcedureIsValid(index_getprocid(bdesc->bd_index, attno, + procnum))) + { + fmgr_info_copy(&opaque->extra_procinfos[basenum], + index_getprocinfo(bdesc->bd_index, attno, procnum), + bdesc->bd_context); + } + else + { + opaque->extra_proc_missing[basenum] = true; + return NULL; + } + } + + return &opaque->extra_procinfos[basenum]; +} + +Datum +brin_bloom_options(PG_FUNCTION_ARGS) +{ + Datum raw_options = PG_GETARG_DATUM(0); + bool validate = PG_GETARG_BOOL(1); + relopt_real opts[] = { + { + {"n_distinct_per_range", "xxx", 0, 0, 20, RELOPT_TYPE_REAL }, + -0.1, -1.0, INT_MAX + }, + { + {"false_positive_rate", "xxx", 0, 0, 19, RELOPT_TYPE_REAL }, + 0.01, 0.001, 1.0 + } + }; + + relopt_gen *optgen[] = { &opts[0].gen, &opts[1].gen }; + int offsets[] = { offsetof(BloomOptions, nDistinctPerRange), + offsetof(BloomOptions, falsePositiveRate)}; + + BloomOptions *options = + parseAndFillLocalRelOptions(raw_options, optgen, offsets, 2, + sizeof(BloomOptions), validate); + + PG_RETURN_POINTER(options); +} diff --git a/src/include/access/brin.h b/src/include/access/brin.h index 8271404332..37cf154e91 100644 --- a/src/include/access/brin.h +++ b/src/include/access/brin.h @@ -23,6 +23,8 @@ typedef struct BrinOptions int32 vl_len_; /* varlena header (do not touch directly!) */ BlockNumber pagesPerRange; bool autosummarize; + double nDistinctPerRange; /* number of distinct values per range */ + double falsePositiveRate; /* false positive for bloom filter */ } BrinOptions; diff --git a/src/include/access/brin_internal.h b/src/include/access/brin_internal.h index 21a79803b3..633e1ec90e 100644 --- a/src/include/access/brin_internal.h +++ b/src/include/access/brin_internal.h @@ -58,6 +58,10 @@ typedef struct BrinDesc /* total number of Datum entries that are stored on-disk for all columns */ int bd_totalstored; + /* parameters for sizing bloom filter (BRIN bloom opclasses) */ + double bd_nDistinctPerRange; + double bd_falsePositiveRange; + /* per-column info; bd_tupdesc->natts entries long */ BrinOpcInfo *bd_info[FLEXIBLE_ARRAY_MEMBER]; } BrinDesc; diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index cf63eb7d54..90ae219598 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1675,6 +1675,11 @@ amoprighttype => 'bytea', amopstrategy => '5', amopopr => '>(bytea,bytea)', amopmethod => 'brin' }, +# bloom bytea +{ amopfamily => 'brin/bytea_bloom_ops', amoplefttype => 'bytea', + amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', + amopmethod => 'brin' }, + # minmax "char" { amopfamily => 'brin/char_minmax_ops', amoplefttype => 'char', amoprighttype => 'char', amopstrategy => '1', amopopr => '<(char,char)', @@ -1692,6 +1697,11 @@ amoprighttype => 'char', amopstrategy => '5', amopopr => '>(char,char)', amopmethod => 'brin' }, +# bloom "char" +{ amopfamily => 'brin/char_bloom_ops', amoplefttype => 'char', + amoprighttype => 'char', amopstrategy => '1', amopopr => '=(char,char)', + amopmethod => 'brin' }, + # minmax name { amopfamily => 'brin/name_minmax_ops', amoplefttype => 'name', amoprighttype => 'name', amopstrategy => '1', amopopr => '<(name,name)', @@ -1709,6 +1719,11 @@ amoprighttype => 'name', amopstrategy => '5', amopopr => '>(name,name)', amopmethod => 'brin' }, +# bloom name +{ amopfamily => 'brin/name_bloom_ops', amoplefttype => 'name', + amoprighttype => 'name', amopstrategy => '1', amopopr => '=(name,name)', + amopmethod => 'brin' }, + # minmax integer { amopfamily => 'brin/integer_minmax_ops', amoplefttype => 'int8', @@ -1855,6 +1870,44 @@ amoprighttype => 'int8', amopstrategy => '5', amopopr => '>(int4,int8)', amopmethod => 'brin' }, +# bloom integer + +{ amopfamily => 'brin/integer_bloom_ops', amoplefttype => 'int8', + amoprighttype => 'int8', amopstrategy => '1', amopopr => '=(int8,int8)', + amopmethod => 'brin' }, + +{ amopfamily => 'brin/integer_bloom_ops', amoplefttype => 'int8', + amoprighttype => 'int2', amopstrategy => '1', amopopr => '=(int8,int2)', + amopmethod => 'brin' }, + +{ amopfamily => 'brin/integer_bloom_ops', amoplefttype => 'int8', + amoprighttype => 'int4', amopstrategy => '1', amopopr => '=(int8,int4)', + amopmethod => 'brin' }, + +{ amopfamily => 'brin/integer_bloom_ops', amoplefttype => 'int2', + amoprighttype => 'int2', amopstrategy => '1', amopopr => '=(int2,int2)', + amopmethod => 'brin' }, + +{ amopfamily => 'brin/integer_bloom_ops', amoplefttype => 'int2', + amoprighttype => 'int8', amopstrategy => '1', amopopr => '=(int2,int8)', + amopmethod => 'brin' }, + +{ amopfamily => 'brin/integer_bloom_ops', amoplefttype => 'int2', + amoprighttype => 'int4', amopstrategy => '1', amopopr => '=(int2,int4)', + amopmethod => 'brin' }, + +{ amopfamily => 'brin/integer_bloom_ops', amoplefttype => 'int4', + amoprighttype => 'int4', amopstrategy => '1', amopopr => '=(int4,int4)', + amopmethod => 'brin' }, + +{ amopfamily => 'brin/integer_bloom_ops', amoplefttype => 'int4', + amoprighttype => 'int2', amopstrategy => '1', amopopr => '=(int4,int2)', + amopmethod => 'brin' }, + +{ amopfamily => 'brin/integer_bloom_ops', amoplefttype => 'int4', + amoprighttype => 'int8', amopstrategy => '1', amopopr => '=(int4,int8)', + amopmethod => 'brin' }, + # minmax text { amopfamily => 'brin/text_minmax_ops', amoplefttype => 'text', amoprighttype => 'text', amopstrategy => '1', amopopr => '<(text,text)', @@ -1872,6 +1925,11 @@ amoprighttype => 'text', amopstrategy => '5', amopopr => '>(text,text)', amopmethod => 'brin' }, +# bloom text +{ amopfamily => 'brin/text_bloom_ops', amoplefttype => 'text', + amoprighttype => 'text', amopstrategy => '1', amopopr => '=(text,text)', + amopmethod => 'brin' }, + # minmax oid { amopfamily => 'brin/oid_minmax_ops', amoplefttype => 'oid', amoprighttype => 'oid', amopstrategy => '1', amopopr => '<(oid,oid)', @@ -1889,6 +1947,11 @@ amoprighttype => 'oid', amopstrategy => '5', amopopr => '>(oid,oid)', amopmethod => 'brin' }, +# bloom oid +{ amopfamily => 'brin/oid_bloom_ops', amoplefttype => 'oid', + amoprighttype => 'oid', amopstrategy => '1', amopopr => '=(oid,oid)', + amopmethod => 'brin' }, + # minmax tid { amopfamily => 'brin/tid_minmax_ops', amoplefttype => 'tid', amoprighttype => 'tid', amopstrategy => '1', amopopr => '<(tid,tid)', @@ -1972,6 +2035,20 @@ amoprighttype => 'float8', amopstrategy => '5', amopopr => '>(float8,float8)', amopmethod => 'brin' }, +# bloom float +{ amopfamily => 'brin/float_bloom_ops', amoplefttype => 'float4', + amoprighttype => 'float4', amopstrategy => '1', amopopr => '=(float4,float4)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/float_bloom_ops', amoplefttype => 'float4', + amoprighttype => 'float8', amopstrategy => '1', amopopr => '=(float4,float8)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/float_bloom_ops', amoplefttype => 'float8', + amoprighttype => 'float4', amopstrategy => '1', amopopr => '=(float8,float4)', + amopmethod => 'brin' }, +{ amopfamily => 'brin/float_bloom_ops', amoplefttype => 'float8', + amoprighttype => 'float8', amopstrategy => '1', amopopr => '=(float8,float8)', + amopmethod => 'brin' }, + # minmax macaddr { amopfamily => 'brin/macaddr_minmax_ops', amoplefttype => 'macaddr', amoprighttype => 'macaddr', amopstrategy => '1', @@ -1989,6 +2066,11 @@ amoprighttype => 'macaddr', amopstrategy => '5', amopopr => '>(macaddr,macaddr)', amopmethod => 'brin' }, +# bloom macaddr +{ amopfamily => 'brin/macaddr_bloom_ops', amoplefttype => 'macaddr', + amoprighttype => 'macaddr', amopstrategy => '1', + amopopr => '=(macaddr,macaddr)', amopmethod => 'brin' }, + # minmax macaddr8 { amopfamily => 'brin/macaddr8_minmax_ops', amoplefttype => 'macaddr8', amoprighttype => 'macaddr8', amopstrategy => '1', @@ -2006,6 +2088,11 @@ amoprighttype => 'macaddr8', amopstrategy => '5', amopopr => '>(macaddr8,macaddr8)', amopmethod => 'brin' }, +# bloom macaddr8 +{ amopfamily => 'brin/macaddr8_bloom_ops', amoplefttype => 'macaddr8', + amoprighttype => 'macaddr8', amopstrategy => '1', + amopopr => '=(macaddr8,macaddr8)', amopmethod => 'brin' }, + # minmax inet { amopfamily => 'brin/network_minmax_ops', amoplefttype => 'inet', amoprighttype => 'inet', amopstrategy => '1', amopopr => '<(inet,inet)', @@ -2023,6 +2110,11 @@ amoprighttype => 'inet', amopstrategy => '5', amopopr => '>(inet,inet)', amopmethod => 'brin' }, +# bloom inet +{ amopfamily => 'brin/network_bloom_ops', amoplefttype => 'inet', + amoprighttype => 'inet', amopstrategy => '1', amopopr => '=(inet,inet)', + amopmethod => 'brin' }, + # inclusion inet { amopfamily => 'brin/network_inclusion_ops', amoplefttype => 'inet', amoprighttype => 'inet', amopstrategy => '3', amopopr => '&&(inet,inet)', @@ -2060,6 +2152,11 @@ amoprighttype => 'bpchar', amopstrategy => '5', amopopr => '>(bpchar,bpchar)', amopmethod => 'brin' }, +# bloom character +{ amopfamily => 'brin/bpchar_bloom_ops', amoplefttype => 'bpchar', + amoprighttype => 'bpchar', amopstrategy => '1', amopopr => '=(bpchar,bpchar)', + amopmethod => 'brin' }, + # minmax time without time zone { amopfamily => 'brin/time_minmax_ops', amoplefttype => 'time', amoprighttype => 'time', amopstrategy => '1', amopopr => '<(time,time)', @@ -2077,6 +2174,11 @@ amoprighttype => 'time', amopstrategy => '5', amopopr => '>(time,time)', amopmethod => 'brin' }, +# bloom time without time zone +{ amopfamily => 'brin/time_bloom_ops', amoplefttype => 'time', + amoprighttype => 'time', amopstrategy => '1', amopopr => '=(time,time)', + amopmethod => 'brin' }, + # minmax datetime (date, timestamp, timestamptz) { amopfamily => 'brin/datetime_minmax_ops', amoplefttype => 'timestamp', @@ -2223,6 +2325,44 @@ amoprighttype => 'timestamptz', amopstrategy => '5', amopopr => '>(timestamptz,timestamptz)', amopmethod => 'brin' }, +# bloom datetime (date, timestamp, timestamptz) + +{ amopfamily => 'brin/datetime_bloom_ops', amoplefttype => 'timestamp', + amoprighttype => 'timestamp', amopstrategy => '1', + amopopr => '=(timestamp,timestamp)', amopmethod => 'brin' }, + +{ amopfamily => 'brin/datetime_bloom_ops', amoplefttype => 'timestamp', + amoprighttype => 'date', amopstrategy => '1', amopopr => '=(timestamp,date)', + amopmethod => 'brin' }, + +{ amopfamily => 'brin/datetime_bloom_ops', amoplefttype => 'timestamp', + amoprighttype => 'timestamptz', amopstrategy => '1', + amopopr => '=(timestamp,timestamptz)', amopmethod => 'brin' }, + +{ amopfamily => 'brin/datetime_bloom_ops', amoplefttype => 'date', + amoprighttype => 'date', amopstrategy => '1', amopopr => '=(date,date)', + amopmethod => 'brin' }, + +{ amopfamily => 'brin/datetime_bloom_ops', amoplefttype => 'date', + amoprighttype => 'timestamp', amopstrategy => '1', + amopopr => '=(date,timestamp)', amopmethod => 'brin' }, + +{ amopfamily => 'brin/datetime_bloom_ops', amoplefttype => 'date', + amoprighttype => 'timestamptz', amopstrategy => '1', + amopopr => '=(date,timestamptz)', amopmethod => 'brin' }, + +{ amopfamily => 'brin/datetime_bloom_ops', amoplefttype => 'timestamptz', + amoprighttype => 'date', amopstrategy => '1', + amopopr => '=(timestamptz,date)', amopmethod => 'brin' }, + +{ amopfamily => 'brin/datetime_bloom_ops', amoplefttype => 'timestamptz', + amoprighttype => 'timestamp', amopstrategy => '1', + amopopr => '=(timestamptz,timestamp)', amopmethod => 'brin' }, + +{ amopfamily => 'brin/datetime_bloom_ops', amoplefttype => 'timestamptz', + amoprighttype => 'timestamptz', amopstrategy => '1', + amopopr => '=(timestamptz,timestamptz)', amopmethod => 'brin' }, + # minmax interval { amopfamily => 'brin/interval_minmax_ops', amoplefttype => 'interval', amoprighttype => 'interval', amopstrategy => '1', @@ -2240,6 +2380,11 @@ amoprighttype => 'interval', amopstrategy => '5', amopopr => '>(interval,interval)', amopmethod => 'brin' }, +# bloom interval +{ amopfamily => 'brin/interval_bloom_ops', amoplefttype => 'interval', + amoprighttype => 'interval', amopstrategy => '1', + amopopr => '=(interval,interval)', amopmethod => 'brin' }, + # minmax time with time zone { amopfamily => 'brin/timetz_minmax_ops', amoplefttype => 'timetz', amoprighttype => 'timetz', amopstrategy => '1', amopopr => '<(timetz,timetz)', @@ -2257,6 +2402,11 @@ amoprighttype => 'timetz', amopstrategy => '5', amopopr => '>(timetz,timetz)', amopmethod => 'brin' }, +# bloom time with time zone +{ amopfamily => 'brin/timetz_bloom_ops', amoplefttype => 'timetz', + amoprighttype => 'timetz', amopstrategy => '1', amopopr => '=(timetz,timetz)', + amopmethod => 'brin' }, + # minmax bit { amopfamily => 'brin/bit_minmax_ops', amoplefttype => 'bit', amoprighttype => 'bit', amopstrategy => '1', amopopr => '<(bit,bit)', @@ -2308,6 +2458,11 @@ amoprighttype => 'numeric', amopstrategy => '5', amopopr => '>(numeric,numeric)', amopmethod => 'brin' }, +# bloom numeric +{ amopfamily => 'brin/numeric_bloom_ops', amoplefttype => 'numeric', + amoprighttype => 'numeric', amopstrategy => '1', + amopopr => '=(numeric,numeric)', amopmethod => 'brin' }, + # minmax uuid { amopfamily => 'brin/uuid_minmax_ops', amoplefttype => 'uuid', amoprighttype => 'uuid', amopstrategy => '1', amopopr => '<(uuid,uuid)', @@ -2325,6 +2480,11 @@ amoprighttype => 'uuid', amopstrategy => '5', amopopr => '>(uuid,uuid)', amopmethod => 'brin' }, +# bloom uuid +{ amopfamily => 'brin/uuid_bloom_ops', amoplefttype => 'uuid', + amoprighttype => 'uuid', amopstrategy => '1', amopopr => '=(uuid,uuid)', + amopmethod => 'brin' }, + # inclusion range types { amopfamily => 'brin/range_inclusion_ops', amoplefttype => 'anyrange', amoprighttype => 'anyrange', amopstrategy => '1', @@ -2386,6 +2546,11 @@ amoprighttype => 'pg_lsn', amopstrategy => '5', amopopr => '>(pg_lsn,pg_lsn)', amopmethod => 'brin' }, +# bloom pg_lsn +{ amopfamily => 'brin/pg_lsn_bloom_ops', amoplefttype => 'pg_lsn', + amoprighttype => 'pg_lsn', amopstrategy => '1', amopopr => '=(pg_lsn,pg_lsn)', + amopmethod => 'brin' }, + # inclusion box { amopfamily => 'brin/box_inclusion_ops', amoplefttype => 'box', amoprighttype => 'box', amopstrategy => '1', amopopr => '<<(box,box)', diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 5ceee11ab1..c7199f0a91 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -697,6 +697,24 @@ { amprocfamily => 'brin/bytea_minmax_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom bytea +{ amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/bytea_bloom_ops', amproclefttype => 'bytea', + amprocrighttype => 'bytea', amprocnum => '11', amproc => 'hashvarlena' }, + # minmax "char" { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '1', @@ -710,6 +728,24 @@ { amprocfamily => 'brin/char_minmax_ops', amproclefttype => 'char', amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom "char" +{ amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/char_bloom_ops', amproclefttype => 'char', + amprocrighttype => 'char', amprocnum => '11', amproc => 'hashchar' }, + # minmax name { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '1', @@ -723,6 +759,24 @@ { amprocfamily => 'brin/name_minmax_ops', amproclefttype => 'name', amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom name +{ amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/name_bloom_ops', amproclefttype => 'name', + amprocrighttype => 'name', amprocnum => '11', amproc => 'hashname' }, + # minmax integer: int2, int4, int8 { amprocfamily => 'brin/integer_minmax_ops', amproclefttype => 'int8', amprocrighttype => 'int8', amprocnum => '1', @@ -824,6 +878,58 @@ { amprocfamily => 'brin/integer_minmax_ops', amproclefttype => 'int4', amprocrighttype => 'int2', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom integer: int2, int4, int8 +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int8', + amprocrighttype => 'int8', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int8', + amprocrighttype => 'int8', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int8', + amprocrighttype => 'int8', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int8', + amprocrighttype => 'int8', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int8', + amprocrighttype => 'int8', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int8', + amprocrighttype => 'int8', amprocnum => '11', amproc => 'hashint8' }, + +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int2', + amprocrighttype => 'int2', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int2', + amprocrighttype => 'int2', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int2', + amprocrighttype => 'int2', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int2', + amprocrighttype => 'int2', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int2', + amprocrighttype => 'int2', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int2', + amprocrighttype => 'int2', amprocnum => '11', amproc => 'hashint2' }, + +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int4', + amprocrighttype => 'int4', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int4', + amprocrighttype => 'int4', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int4', + amprocrighttype => 'int4', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int4', + amprocrighttype => 'int4', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int4', + amprocrighttype => 'int4', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/integer_bloom_ops', amproclefttype => 'int4', + amprocrighttype => 'int4', amprocnum => '11', amproc => 'hashint4' }, + # minmax text { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '1', @@ -837,6 +943,24 @@ { amprocfamily => 'brin/text_minmax_ops', amproclefttype => 'text', amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom text +{ amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/text_bloom_ops', amproclefttype => 'text', + amprocrighttype => 'text', amprocnum => '11', amproc => 'hashtext' }, + # minmax oid { amprocfamily => 'brin/oid_minmax_ops', amproclefttype => 'oid', amprocrighttype => 'oid', amprocnum => '1', amproc => 'brin_minmax_opcinfo' }, @@ -849,6 +973,23 @@ { amprocfamily => 'brin/oid_minmax_ops', amproclefttype => 'oid', amprocrighttype => 'oid', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom oid +{ amprocfamily => 'brin/oid_bloom_ops', amproclefttype => 'oid', + amprocrighttype => 'oid', amprocnum => '1', amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/oid_bloom_ops', amproclefttype => 'oid', + amprocrighttype => 'oid', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/oid_bloom_ops', amproclefttype => 'oid', + amprocrighttype => 'oid', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/oid_bloom_ops', amproclefttype => 'oid', + amprocrighttype => 'oid', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/oid_bloom_ops', amproclefttype => 'oid', + amprocrighttype => 'oid', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/oid_bloom_ops', amproclefttype => 'oid', + amprocrighttype => 'oid', amprocnum => '11', amproc => 'hashoid' }, + # minmax tid { amprocfamily => 'brin/tid_minmax_ops', amproclefttype => 'tid', amprocrighttype => 'tid', amprocnum => '1', amproc => 'brin_minmax_opcinfo' }, @@ -911,6 +1052,45 @@ amprocrighttype => 'float4', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom float +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float4', + amprocrighttype => 'float4', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float4', + amprocrighttype => 'float4', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float4', + amprocrighttype => 'float4', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float4', + amprocrighttype => 'float4', amprocnum => '4', + amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float4', + amprocrighttype => 'float4', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float4', + amprocrighttype => 'float4', amprocnum => '11', + amproc => 'hashfloat4' }, + +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float8', + amprocrighttype => 'float8', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float8', + amprocrighttype => 'float8', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float8', + amprocrighttype => 'float8', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float8', + amprocrighttype => 'float8', amprocnum => '4', + amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float8', + amprocrighttype => 'float8', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/float_bloom_ops', amproclefttype => 'float8', + amprocrighttype => 'float8', amprocnum => '11', + amproc => 'hashfloat8' }, + # minmax macaddr { amprocfamily => 'brin/macaddr_minmax_ops', amproclefttype => 'macaddr', amprocrighttype => 'macaddr', amprocnum => '1', @@ -925,6 +1105,26 @@ amprocrighttype => 'macaddr', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom macaddr +{ amprocfamily => 'brin/macaddr_bloom_ops', amproclefttype => 'macaddr', + amprocrighttype => 'macaddr', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/macaddr_bloom_ops', amproclefttype => 'macaddr', + amprocrighttype => 'macaddr', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/macaddr_bloom_ops', amproclefttype => 'macaddr', + amprocrighttype => 'macaddr', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/macaddr_bloom_ops', amproclefttype => 'macaddr', + amprocrighttype => 'macaddr', amprocnum => '4', + amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/macaddr_bloom_ops', amproclefttype => 'macaddr', + amprocrighttype => 'macaddr', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/macaddr_bloom_ops', amproclefttype => 'macaddr', + amprocrighttype => 'macaddr', amprocnum => '11', + amproc => 'hashmacaddr' }, + # minmax macaddr8 { amprocfamily => 'brin/macaddr8_minmax_ops', amproclefttype => 'macaddr8', amprocrighttype => 'macaddr8', amprocnum => '1', @@ -939,6 +1139,26 @@ amprocrighttype => 'macaddr8', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom macaddr8 +{ amprocfamily => 'brin/macaddr8_bloom_ops', amproclefttype => 'macaddr8', + amprocrighttype => 'macaddr8', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/macaddr8_bloom_ops', amproclefttype => 'macaddr8', + amprocrighttype => 'macaddr8', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/macaddr8_bloom_ops', amproclefttype => 'macaddr8', + amprocrighttype => 'macaddr8', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/macaddr8_bloom_ops', amproclefttype => 'macaddr8', + amprocrighttype => 'macaddr8', amprocnum => '4', + amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/macaddr8_bloom_ops', amproclefttype => 'macaddr8', + amprocrighttype => 'macaddr8', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/macaddr8_bloom_ops', amproclefttype => 'macaddr8', + amprocrighttype => 'macaddr8', amprocnum => '11', + amproc => 'hashmacaddr8' }, + # minmax inet { amprocfamily => 'brin/network_minmax_ops', amproclefttype => 'inet', amprocrighttype => 'inet', amprocnum => '1', @@ -952,6 +1172,24 @@ { amprocfamily => 'brin/network_minmax_ops', amproclefttype => 'inet', amprocrighttype => 'inet', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom inet +{ amprocfamily => 'brin/network_bloom_ops', amproclefttype => 'inet', + amprocrighttype => 'inet', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/network_bloom_ops', amproclefttype => 'inet', + amprocrighttype => 'inet', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/network_bloom_ops', amproclefttype => 'inet', + amprocrighttype => 'inet', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/network_bloom_ops', amproclefttype => 'inet', + amprocrighttype => 'inet', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/network_bloom_ops', amproclefttype => 'inet', + amprocrighttype => 'inet', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/network_bloom_ops', amproclefttype => 'inet', + amprocrighttype => 'inet', amprocnum => '11', amproc => 'hashinet' }, + # inclusion inet { amprocfamily => 'brin/network_inclusion_ops', amproclefttype => 'inet', amprocrighttype => 'inet', amprocnum => '1', @@ -986,6 +1224,26 @@ amprocrighttype => 'bpchar', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom character +{ amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '4', + amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/bpchar_bloom_ops', amproclefttype => 'bpchar', + amprocrighttype => 'bpchar', amprocnum => '11', + amproc => 'hashchar' }, + # minmax time without time zone { amprocfamily => 'brin/time_minmax_ops', amproclefttype => 'time', amprocrighttype => 'time', amprocnum => '1', @@ -999,6 +1257,24 @@ { amprocfamily => 'brin/time_minmax_ops', amproclefttype => 'time', amprocrighttype => 'time', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom time without time zone +{ amprocfamily => 'brin/time_bloom_ops', amproclefttype => 'time', + amprocrighttype => 'time', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/time_bloom_ops', amproclefttype => 'time', + amprocrighttype => 'time', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/time_bloom_ops', amproclefttype => 'time', + amprocrighttype => 'time', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/time_bloom_ops', amproclefttype => 'time', + amprocrighttype => 'time', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/time_bloom_ops', amproclefttype => 'time', + amprocrighttype => 'time', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/time_bloom_ops', amproclefttype => 'time', + amprocrighttype => 'time', amprocnum => '11', amproc => 'time_hash' }, + # minmax datetime (date, timestamp, timestamptz) { amprocfamily => 'brin/datetime_minmax_ops', amproclefttype => 'timestamp', amprocrighttype => 'timestamp', amprocnum => '1', @@ -1106,6 +1382,62 @@ amprocrighttype => 'timestamptz', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom datetime (date, timestamp, timestamptz) +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamp', + amprocrighttype => 'timestamp', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamp', + amprocrighttype => 'timestamp', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamp', + amprocrighttype => 'timestamp', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamp', + amprocrighttype => 'timestamp', amprocnum => '4', + amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamp', + amprocrighttype => 'timestamp', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamp', + amprocrighttype => 'timestamp', amprocnum => '11', + amproc => 'timestamp_hash' }, + +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamptz', + amprocrighttype => 'timestamptz', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamptz', + amprocrighttype => 'timestamptz', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamptz', + amprocrighttype => 'timestamptz', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamptz', + amprocrighttype => 'timestamptz', amprocnum => '4', + amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamptz', + amprocrighttype => 'timestamptz', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'timestamptz', + amprocrighttype => 'timestamptz', amprocnum => '11', + amproc => 'timestamp_hash' }, + +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'date', + amprocrighttype => 'date', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'date', + amprocrighttype => 'date', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'date', + amprocrighttype => 'date', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'date', + amprocrighttype => 'date', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'date', + amprocrighttype => 'date', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/datetime_bloom_ops', amproclefttype => 'date', + amprocrighttype => 'date', amprocnum => '11', amproc => 'hashint4' }, + # minmax interval { amprocfamily => 'brin/interval_minmax_ops', amproclefttype => 'interval', amprocrighttype => 'interval', amprocnum => '1', @@ -1120,6 +1452,26 @@ amprocrighttype => 'interval', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom interval +{ amprocfamily => 'brin/interval_bloom_ops', amproclefttype => 'interval', + amprocrighttype => 'interval', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/interval_bloom_ops', amproclefttype => 'interval', + amprocrighttype => 'interval', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/interval_bloom_ops', amproclefttype => 'interval', + amprocrighttype => 'interval', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/interval_bloom_ops', amproclefttype => 'interval', + amprocrighttype => 'interval', amprocnum => '4', + amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/interval_bloom_ops', amproclefttype => 'interval', + amprocrighttype => 'interval', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/interval_bloom_ops', amproclefttype => 'interval', + amprocrighttype => 'interval', amprocnum => '11', + amproc => 'interval_hash' }, + # minmax time with time zone { amprocfamily => 'brin/timetz_minmax_ops', amproclefttype => 'timetz', amprocrighttype => 'timetz', amprocnum => '1', @@ -1134,6 +1486,26 @@ amprocrighttype => 'timetz', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom time with time zone +{ amprocfamily => 'brin/timetz_bloom_ops', amproclefttype => 'timetz', + amprocrighttype => 'timetz', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/timetz_bloom_ops', amproclefttype => 'timetz', + amprocrighttype => 'timetz', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/timetz_bloom_ops', amproclefttype => 'timetz', + amprocrighttype => 'timetz', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/timetz_bloom_ops', amproclefttype => 'timetz', + amprocrighttype => 'timetz', amprocnum => '4', + amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/timetz_bloom_ops', amproclefttype => 'timetz', + amprocrighttype => 'timetz', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/timetz_bloom_ops', amproclefttype => 'timetz', + amprocrighttype => 'timetz', amprocnum => '11', + amproc => 'timetz_hash' }, + # minmax bit { amprocfamily => 'brin/bit_minmax_ops', amproclefttype => 'bit', amprocrighttype => 'bit', amprocnum => '1', amproc => 'brin_minmax_opcinfo' }, @@ -1174,6 +1546,26 @@ amprocrighttype => 'numeric', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom numeric +{ amprocfamily => 'brin/numeric_bloom_ops', amproclefttype => 'numeric', + amprocrighttype => 'numeric', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/numeric_bloom_ops', amproclefttype => 'numeric', + amprocrighttype => 'numeric', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/numeric_bloom_ops', amproclefttype => 'numeric', + amprocrighttype => 'numeric', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/numeric_bloom_ops', amproclefttype => 'numeric', + amprocrighttype => 'numeric', amprocnum => '4', + amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/numeric_bloom_ops', amproclefttype => 'numeric', + amprocrighttype => 'numeric', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/numeric_bloom_ops', amproclefttype => 'numeric', + amprocrighttype => 'numeric', amprocnum => '11', + amproc => 'hash_numeric' }, + # minmax uuid { amprocfamily => 'brin/uuid_minmax_ops', amproclefttype => 'uuid', amprocrighttype => 'uuid', amprocnum => '1', @@ -1187,6 +1579,24 @@ { amprocfamily => 'brin/uuid_minmax_ops', amproclefttype => 'uuid', amprocrighttype => 'uuid', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom uuid +{ amprocfamily => 'brin/uuid_bloom_ops', amproclefttype => 'uuid', + amprocrighttype => 'uuid', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/uuid_bloom_ops', amproclefttype => 'uuid', + amprocrighttype => 'uuid', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/uuid_bloom_ops', amproclefttype => 'uuid', + amprocrighttype => 'uuid', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/uuid_bloom_ops', amproclefttype => 'uuid', + amprocrighttype => 'uuid', amprocnum => '4', amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/uuid_bloom_ops', amproclefttype => 'uuid', + amprocrighttype => 'uuid', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/uuid_bloom_ops', amproclefttype => 'uuid', + amprocrighttype => 'uuid', amprocnum => '11', amproc => 'uuid_hash' }, + # inclusion range types { amprocfamily => 'brin/range_inclusion_ops', amproclefttype => 'anyrange', amprocrighttype => 'anyrange', amprocnum => '1', @@ -1222,6 +1632,26 @@ amprocrighttype => 'pg_lsn', amprocnum => '4', amproc => 'brin_minmax_union' }, +# bloom pg_lsn +{ amprocfamily => 'brin/pg_lsn_bloom_ops', amproclefttype => 'pg_lsn', + amprocrighttype => 'pg_lsn', amprocnum => '1', + amproc => 'brin_bloom_opcinfo' }, +{ amprocfamily => 'brin/pg_lsn_bloom_ops', amproclefttype => 'pg_lsn', + amprocrighttype => 'pg_lsn', amprocnum => '2', + amproc => 'brin_bloom_add_value' }, +{ amprocfamily => 'brin/pg_lsn_bloom_ops', amproclefttype => 'pg_lsn', + amprocrighttype => 'pg_lsn', amprocnum => '3', + amproc => 'brin_bloom_consistent' }, +{ amprocfamily => 'brin/pg_lsn_bloom_ops', amproclefttype => 'pg_lsn', + amprocrighttype => 'pg_lsn', amprocnum => '4', + amproc => 'brin_bloom_union' }, +{ amprocfamily => 'brin/pg_lsn_bloom_ops', amproclefttype => 'pg_lsn', + amprocrighttype => 'pg_lsn', amprocnum => '5', + amproc => 'brin_bloom_options' }, +{ amprocfamily => 'brin/pg_lsn_bloom_ops', amproclefttype => 'pg_lsn', + amprocrighttype => 'pg_lsn', amprocnum => '11', + amproc => 'pg_lsn_hash' }, + # inclusion box { amprocfamily => 'brin/box_inclusion_ops', amproclefttype => 'box', amprocrighttype => 'box', amprocnum => '1', diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index fdfea85efe..ebc35fd98c 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -250,67 +250,127 @@ { opcmethod => 'brin', opcname => 'bytea_minmax_ops', opcfamily => 'brin/bytea_minmax_ops', opcintype => 'bytea', opckeytype => 'bytea' }, +{ opcmethod => 'brin', opcname => 'bytea_bloom_ops', + opcfamily => 'brin/bytea_bloom_ops', opcintype => 'bytea', + opckeytype => 'bytea', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'char_minmax_ops', opcfamily => 'brin/char_minmax_ops', opcintype => 'char', opckeytype => 'char' }, +{ opcmethod => 'brin', opcname => 'char_bloom_ops', + opcfamily => 'brin/char_bloom_ops', opcintype => 'char', + opckeytype => 'char', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'name_minmax_ops', opcfamily => 'brin/name_minmax_ops', opcintype => 'name', opckeytype => 'name' }, +{ opcmethod => 'brin', opcname => 'name_bloom_ops', + opcfamily => 'brin/name_bloom_ops', opcintype => 'name', + opckeytype => 'name', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'int8_minmax_ops', opcfamily => 'brin/integer_minmax_ops', opcintype => 'int8', opckeytype => 'int8' }, +{ opcmethod => 'brin', opcname => 'int8_bloom_ops', + opcfamily => 'brin/integer_bloom_ops', opcintype => 'int8', + opckeytype => 'int8', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'int2_minmax_ops', opcfamily => 'brin/integer_minmax_ops', opcintype => 'int2', opckeytype => 'int2' }, +{ opcmethod => 'brin', opcname => 'int2_bloom_ops', + opcfamily => 'brin/integer_bloom_ops', opcintype => 'int2', + opckeytype => 'int2', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'int4_minmax_ops', opcfamily => 'brin/integer_minmax_ops', opcintype => 'int4', opckeytype => 'int4' }, +{ opcmethod => 'brin', opcname => 'int4_bloom_ops', + opcfamily => 'brin/integer_bloom_ops', opcintype => 'int4', + opckeytype => 'int4', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'text_minmax_ops', opcfamily => 'brin/text_minmax_ops', opcintype => 'text', opckeytype => 'text' }, +{ opcmethod => 'brin', opcname => 'text_bloom_ops', + opcfamily => 'brin/text_bloom_ops', opcintype => 'text', + opckeytype => 'text', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'oid_minmax_ops', opcfamily => 'brin/oid_minmax_ops', opcintype => 'oid', opckeytype => 'oid' }, +{ opcmethod => 'brin', opcname => 'oid_bloom_ops', + opcfamily => 'brin/oid_bloom_ops', opcintype => 'oid', + opckeytype => 'oid', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'tid_minmax_ops', opcfamily => 'brin/tid_minmax_ops', opcintype => 'tid', opckeytype => 'tid' }, { opcmethod => 'brin', opcname => 'float4_minmax_ops', opcfamily => 'brin/float_minmax_ops', opcintype => 'float4', opckeytype => 'float4' }, +{ opcmethod => 'brin', opcname => 'float4_bloom_ops', + opcfamily => 'brin/float_bloom_ops', opcintype => 'float4', + opckeytype => 'float4', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'float8_minmax_ops', opcfamily => 'brin/float_minmax_ops', opcintype => 'float8', opckeytype => 'float8' }, +{ opcmethod => 'brin', opcname => 'float8_bloom_ops', + opcfamily => 'brin/float_bloom_ops', opcintype => 'float8', + opckeytype => 'float8', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'macaddr_minmax_ops', opcfamily => 'brin/macaddr_minmax_ops', opcintype => 'macaddr', opckeytype => 'macaddr' }, +{ opcmethod => 'brin', opcname => 'macaddr_bloom_ops', + opcfamily => 'brin/macaddr_bloom_ops', opcintype => 'macaddr', + opckeytype => 'macaddr', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'macaddr8_minmax_ops', opcfamily => 'brin/macaddr8_minmax_ops', opcintype => 'macaddr8', opckeytype => 'macaddr8' }, +{ opcmethod => 'brin', opcname => 'macaddr8_bloom_ops', + opcfamily => 'brin/macaddr8_bloom_ops', opcintype => 'macaddr8', + opckeytype => 'macaddr8', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'inet_minmax_ops', opcfamily => 'brin/network_minmax_ops', opcintype => 'inet', opcdefault => 'f', opckeytype => 'inet' }, +{ opcmethod => 'brin', opcname => 'inet_bloom_ops', + opcfamily => 'brin/network_bloom_ops', opcintype => 'inet', + opcdefault => 'f', opckeytype => 'inet', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'inet_inclusion_ops', opcfamily => 'brin/network_inclusion_ops', opcintype => 'inet', opckeytype => 'inet' }, { opcmethod => 'brin', opcname => 'bpchar_minmax_ops', opcfamily => 'brin/bpchar_minmax_ops', opcintype => 'bpchar', opckeytype => 'bpchar' }, +{ opcmethod => 'brin', opcname => 'bpchar_bloom_ops', + opcfamily => 'brin/bpchar_bloom_ops', opcintype => 'bpchar', + opckeytype => 'bpchar', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'time_minmax_ops', opcfamily => 'brin/time_minmax_ops', opcintype => 'time', opckeytype => 'time' }, +{ opcmethod => 'brin', opcname => 'time_bloom_ops', + opcfamily => 'brin/time_bloom_ops', opcintype => 'time', + opckeytype => 'time', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'date_minmax_ops', opcfamily => 'brin/datetime_minmax_ops', opcintype => 'date', opckeytype => 'date' }, +{ opcmethod => 'brin', opcname => 'date_bloom_ops', + opcfamily => 'brin/datetime_bloom_ops', opcintype => 'date', + opckeytype => 'date', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'timestamp_minmax_ops', opcfamily => 'brin/datetime_minmax_ops', opcintype => 'timestamp', opckeytype => 'timestamp' }, +{ opcmethod => 'brin', opcname => 'timestamp_bloom_ops', + opcfamily => 'brin/datetime_bloom_ops', opcintype => 'timestamp', + opckeytype => 'timestamp', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'timestamptz_minmax_ops', opcfamily => 'brin/datetime_minmax_ops', opcintype => 'timestamptz', opckeytype => 'timestamptz' }, +{ opcmethod => 'brin', opcname => 'timestamptz_bloom_ops', + opcfamily => 'brin/datetime_bloom_ops', opcintype => 'timestamptz', + opckeytype => 'timestamptz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'interval_minmax_ops', opcfamily => 'brin/interval_minmax_ops', opcintype => 'interval', opckeytype => 'interval' }, +{ opcmethod => 'brin', opcname => 'interval_bloom_ops', + opcfamily => 'brin/interval_bloom_ops', opcintype => 'interval', + opckeytype => 'interval', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'timetz_minmax_ops', opcfamily => 'brin/timetz_minmax_ops', opcintype => 'timetz', opckeytype => 'timetz' }, +{ opcmethod => 'brin', opcname => 'timetz_bloom_ops', + opcfamily => 'brin/timetz_bloom_ops', opcintype => 'timetz', + opckeytype => 'timetz', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'bit_minmax_ops', opcfamily => 'brin/bit_minmax_ops', opcintype => 'bit', opckeytype => 'bit' }, { opcmethod => 'brin', opcname => 'varbit_minmax_ops', @@ -319,18 +379,27 @@ { opcmethod => 'brin', opcname => 'numeric_minmax_ops', opcfamily => 'brin/numeric_minmax_ops', opcintype => 'numeric', opckeytype => 'numeric' }, +{ opcmethod => 'brin', opcname => 'numeric_bloom_ops', + opcfamily => 'brin/numeric_bloom_ops', opcintype => 'numeric', + opckeytype => 'numeric', opcdefault => 'f' }, # no brin opclass for record, anyarray { opcmethod => 'brin', opcname => 'uuid_minmax_ops', opcfamily => 'brin/uuid_minmax_ops', opcintype => 'uuid', opckeytype => 'uuid' }, +{ opcmethod => 'brin', opcname => 'uuid_bloom_ops', + opcfamily => 'brin/uuid_bloom_ops', opcintype => 'uuid', + opckeytype => 'uuid', opcdefault => 'f' }, { opcmethod => 'brin', opcname => 'range_inclusion_ops', opcfamily => 'brin/range_inclusion_ops', opcintype => 'anyrange', opckeytype => 'anyrange' }, { opcmethod => 'brin', opcname => 'pg_lsn_minmax_ops', opcfamily => 'brin/pg_lsn_minmax_ops', opcintype => 'pg_lsn', opckeytype => 'pg_lsn' }, +{ opcmethod => 'brin', opcname => 'pg_lsn_bloom_ops', + opcfamily => 'brin/pg_lsn_bloom_ops', opcintype => 'pg_lsn', + opckeytype => 'pg_lsn', opcdefault => 'f' }, # no brin opclass for enum, tsvector, tsquery, jsonb diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index 41e40d657a..fd3d659b68 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -176,50 +176,86 @@ opfmethod => 'gin', opfname => 'jsonb_path_ops' }, { oid => '4054', opfmethod => 'brin', opfname => 'integer_minmax_ops' }, +{ oid => '5024', + opfmethod => 'brin', opfname => 'integer_bloom_ops' }, { oid => '4055', opfmethod => 'brin', opfname => 'numeric_minmax_ops' }, { oid => '4056', opfmethod => 'brin', opfname => 'text_minmax_ops' }, +{ oid => '5027', + opfmethod => 'brin', opfname => 'text_bloom_ops' }, +{ oid => '5045', + opfmethod => 'brin', opfname => 'numeric_bloom_ops' }, { oid => '4058', opfmethod => 'brin', opfname => 'timetz_minmax_ops' }, +{ oid => '5042', + opfmethod => 'brin', opfname => 'timetz_bloom_ops' }, { oid => '4059', opfmethod => 'brin', opfname => 'datetime_minmax_ops' }, +{ oid => '5038', + opfmethod => 'brin', opfname => 'datetime_bloom_ops' }, { oid => '4062', opfmethod => 'brin', opfname => 'char_minmax_ops' }, +{ oid => '5022', + opfmethod => 'brin', opfname => 'char_bloom_ops' }, { oid => '4064', opfmethod => 'brin', opfname => 'bytea_minmax_ops' }, +{ oid => '5048', + opfmethod => 'brin', opfname => 'bytea_bloom_ops' }, { oid => '4065', opfmethod => 'brin', opfname => 'name_minmax_ops' }, +{ oid => '5023', + opfmethod => 'brin', opfname => 'name_bloom_ops' }, { oid => '4068', opfmethod => 'brin', opfname => 'oid_minmax_ops' }, +{ oid => '5049', + opfmethod => 'brin', opfname => 'oid_bloom_ops' }, { oid => '4069', opfmethod => 'brin', opfname => 'tid_minmax_ops' }, { oid => '4070', opfmethod => 'brin', opfname => 'float_minmax_ops' }, +{ oid => '5050', + opfmethod => 'brin', opfname => 'float_bloom_ops' }, { oid => '4074', opfmethod => 'brin', opfname => 'macaddr_minmax_ops' }, +{ oid => '5033', + opfmethod => 'brin', opfname => 'macaddr_bloom_ops' }, { oid => '4109', opfmethod => 'brin', opfname => 'macaddr8_minmax_ops' }, +{ oid => '5034', + opfmethod => 'brin', opfname => 'macaddr8_bloom_ops' }, { oid => '4075', opfmethod => 'brin', opfname => 'network_minmax_ops' }, { oid => '4102', opfmethod => 'brin', opfname => 'network_inclusion_ops' }, +{ oid => '5035', + opfmethod => 'brin', opfname => 'network_bloom_ops' }, { oid => '4076', opfmethod => 'brin', opfname => 'bpchar_minmax_ops' }, +{ oid => '5036', + opfmethod => 'brin', opfname => 'bpchar_bloom_ops' }, { oid => '4077', opfmethod => 'brin', opfname => 'time_minmax_ops' }, +{ oid => '5037', + opfmethod => 'brin', opfname => 'time_bloom_ops' }, { oid => '4078', opfmethod => 'brin', opfname => 'interval_minmax_ops' }, +{ oid => '5041', + opfmethod => 'brin', opfname => 'interval_bloom_ops' }, { oid => '4079', opfmethod => 'brin', opfname => 'bit_minmax_ops' }, { oid => '4080', opfmethod => 'brin', opfname => 'varbit_minmax_ops' }, { oid => '4081', opfmethod => 'brin', opfname => 'uuid_minmax_ops' }, +{ oid => '5046', + opfmethod => 'brin', opfname => 'uuid_bloom_ops' }, { oid => '4103', opfmethod => 'brin', opfname => 'range_inclusion_ops' }, { oid => '4082', opfmethod => 'brin', opfname => 'pg_lsn_minmax_ops' }, +{ oid => '5047', + opfmethod => 'brin', opfname => 'pg_lsn_bloom_ops' }, { oid => '4104', opfmethod => 'brin', opfname => 'box_inclusion_ops' }, { oid => '5000', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 3fb4073367..0a1a83807b 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -7914,6 +7914,26 @@ proargtypes => 'internal internal internal internal', prosrc => 'brin_inclusion_union' }, +# BRIN bloom +{ oid => '5039', descr => 'BRIN bloom support', + proname => 'brin_bloom_opcinfo', prorettype => 'internal', + proargtypes => 'internal', prosrc => 'brin_bloom_opcinfo' }, +{ oid => '5040', descr => 'BRIN bloom support', + proname => 'brin_bloom_add_value', prorettype => 'bool', + proargtypes => 'internal internal internal internal internal', + prosrc => 'brin_bloom_add_value' }, +{ oid => '5043', descr => 'BRIN bloom support', + proname => 'brin_bloom_consistent', prorettype => 'bool', + proargtypes => 'internal internal internal int4 internal', + prosrc => 'brin_bloom_consistent' }, +{ oid => '5044', descr => 'BRIN bloom support', + proname => 'brin_bloom_union', prorettype => 'bool', + proargtypes => 'internal internal internal internal', + prosrc => 'brin_bloom_union' }, +{ oid => '6016', descr => 'BRIN bloom support', + proname => 'brin_bloom_options', prorettype => 'internal', + proargtypes => 'internal bool', prosrc => 'brin_bloom_options' }, + # userlock replacements { oid => '2880', descr => 'obtain exclusive advisory lock', proname => 'pg_advisory_lock', provolatile => 'v', proparallel => 'u', diff --git a/src/test/regress/expected/brin_bloom.out b/src/test/regress/expected/brin_bloom.out new file mode 100644 index 0000000000..d58a4a483c --- /dev/null +++ b/src/test/regress/expected/brin_bloom.out @@ -0,0 +1,456 @@ +CREATE TABLE brintest_bloom (byteacol bytea, + charcol "char", + namecol name, + int8col bigint, + int2col smallint, + int4col integer, + textcol text, + oidcol oid, + float4col real, + float8col double precision, + macaddrcol macaddr, + inetcol inet, + cidrcol cidr, + bpcharcol character, + datecol date, + timecol time without time zone, + timestampcol timestamp without time zone, + timestamptzcol timestamp with time zone, + intervalcol interval, + timetzcol time with time zone, + numericcol numeric, + uuidcol uuid, + lsncol pg_lsn +) WITH (fillfactor=10); +INSERT INTO brintest_bloom SELECT + repeat(stringu1, 8)::bytea, + substr(stringu1, 1, 1)::"char", + stringu1::name, 142857 * tenthous, + thousand, + twothousand, + repeat(stringu1, 8), + unique1::oid, + (four + 1.0)/(hundred+1), + odd::float8 / (tenthous + 1), + format('%s:00:%s:00:%s:00', to_hex(odd), to_hex(even), to_hex(hundred))::macaddr, + inet '10.2.3.4/24' + tenthous, + cidr '10.2.3/24' + tenthous, + substr(stringu1, 1, 1)::bpchar, + date '1995-08-15' + tenthous, + time '01:20:30' + thousand * interval '18.5 second', + timestamp '1942-07-23 03:05:09' + tenthous * interval '36.38 hours', + timestamptz '1972-10-10 03:00' + thousand * interval '1 hour', + justify_days(justify_hours(tenthous * interval '12 minutes')), + timetz '01:30:20+02' + hundred * interval '15 seconds', + tenthous::numeric(36,30) * fivethous * even / (hundred + 1), + format('%s%s-%s-%s-%s-%s%s%s', to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'))::uuid, + format('%s/%s%s', odd, even, tenthous)::pg_lsn +FROM tenk1 ORDER BY unique2 LIMIT 100; +-- throw in some NULL's and different values +INSERT INTO brintest_bloom (inetcol, cidrcol) SELECT + inet 'fe80::6e40:8ff:fea9:8c46' + tenthous, + cidr 'fe80::6e40:8ff:fea9:8c46' + tenthous +FROM tenk1 ORDER BY thousand, tenthous LIMIT 25; +-- test bloom specific index options +-- ndistinct must be >= -1.0 +CREATE INDEX brinidx_bloom ON brintest_bloom USING brin ( + byteacol bytea_bloom_ops(n_distinct_per_range = -1.1) +); +ERROR: value -1.1 out of bounds for option "n_distinct_per_range" +DETAIL: Valid values are between "-1.000000" and "2147483647.000000". +-- false_positive_rate must be between 0.001 and 1.0 +CREATE INDEX brinidx_bloom ON brintest_bloom USING brin ( + byteacol bytea_bloom_ops(false_positive_rate = 0.0) +); +ERROR: value 0.0 out of bounds for option "false_positive_rate" +DETAIL: Valid values are between "0.001000" and "1.000000". +CREATE INDEX brinidx_bloom ON brintest_bloom USING brin ( + byteacol bytea_bloom_ops(false_positive_rate = 1.01) +); +ERROR: value 1.01 out of bounds for option "false_positive_rate" +DETAIL: Valid values are between "0.001000" and "1.000000". +CREATE INDEX brinidx_bloom ON brintest_bloom USING brin ( + byteacol bytea_bloom_ops, + charcol char_bloom_ops, + namecol name_bloom_ops, + int8col int8_bloom_ops, + int2col int2_bloom_ops, + int4col int4_bloom_ops, + textcol text_bloom_ops, + oidcol oid_bloom_ops, + float4col float4_bloom_ops, + float8col float8_bloom_ops, + macaddrcol macaddr_bloom_ops, + inetcol inet_bloom_ops, + cidrcol inet_bloom_ops, + bpcharcol bpchar_bloom_ops, + datecol date_bloom_ops, + timecol time_bloom_ops, + timestampcol timestamp_bloom_ops, + timestamptzcol timestamptz_bloom_ops, + intervalcol interval_bloom_ops, + timetzcol timetz_bloom_ops, + numericcol numeric_bloom_ops, + uuidcol uuid_bloom_ops, + lsncol pg_lsn_bloom_ops +) with (pages_per_range = 1); +CREATE TABLE brinopers_bloom (colname name, typ text, + op text[], value text[], matches int[], + check (cardinality(op) = cardinality(value)), + check (cardinality(op) = cardinality(matches))); +INSERT INTO brinopers_bloom VALUES + ('byteacol', 'bytea', + '{=}', + '{BNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAA}', + '{1}'), + ('charcol', '"char"', + '{=}', + '{M}', + '{6}'), + ('namecol', 'name', + '{=}', + '{MAAAAA}', + '{2}'), + ('int2col', 'int2', + '{=}', + '{800}', + '{1}'), + ('int2col', 'int4', + '{=}', + '{800}', + '{1}'), + ('int2col', 'int8', + '{=}', + '{800}', + '{1}'), + ('int4col', 'int2', + '{=}', + '{800}', + '{1}'), + ('int4col', 'int4', + '{=}', + '{800}', + '{1}'), + ('int4col', 'int8', + '{=}', + '{800}', + '{1}'), + ('int8col', 'int8', + '{=}', + '{1257141600}', + '{1}'), + ('textcol', 'text', + '{=}', + '{BNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAA}', + '{1}'), + ('oidcol', 'oid', + '{=}', + '{8800}', + '{1}'), + ('float4col', 'float4', + '{=}', + '{1}', + '{4}'), + ('float4col', 'float8', + '{=}', + '{1}', + '{4}'), + ('float8col', 'float4', + '{=}', + '{0}', + '{1}'), + ('float8col', 'float8', + '{=}', + '{0}', + '{1}'), + ('macaddrcol', 'macaddr', + '{=}', + '{2c:00:2d:00:16:00}', + '{2}'), + ('inetcol', 'inet', + '{=}', + '{10.2.14.231/24}', + '{1}'), + ('inetcol', 'cidr', + '{=}', + '{fe80::6e40:8ff:fea9:8c46}', + '{1}'), + ('cidrcol', 'inet', + '{=}', + '{10.2.14/24}', + '{2}'), + ('cidrcol', 'inet', + '{=}', + '{fe80::6e40:8ff:fea9:8c46}', + '{1}'), + ('cidrcol', 'cidr', + '{=}', + '{10.2.14/24}', + '{2}'), + ('cidrcol', 'cidr', + '{=}', + '{fe80::6e40:8ff:fea9:8c46}', + '{1}'), + ('bpcharcol', 'bpchar', + '{=}', + '{W}', + '{6}'), + ('datecol', 'date', + '{=}', + '{2009-12-01}', + '{1}'), + ('timecol', 'time', + '{=}', + '{02:28:57}', + '{1}'), + ('timestampcol', 'timestamp', + '{=}', + '{1964-03-24 19:26:45}', + '{1}'), + ('timestampcol', 'timestamptz', + '{=}', + '{1964-03-24 19:26:45}', + '{1}'), + ('timestamptzcol', 'timestamptz', + '{=}', + '{1972-10-19 09:00:00-07}', + '{1}'), + ('intervalcol', 'interval', + '{=}', + '{1 mons 13 days 12:24}', + '{1}'), + ('timetzcol', 'timetz', + '{=}', + '{01:35:50+02}', + '{2}'), + ('numericcol', 'numeric', + '{=}', + '{2268164.347826086956521739130434782609}', + '{1}'), + ('uuidcol', 'uuid', + '{=}', + '{52225222-5222-5222-5222-522252225222}', + '{1}'), + ('lsncol', 'pg_lsn', + '{=, IS, IS NOT}', + '{44/455222, NULL, NULL}', + '{1, 25, 100}'); +DO $x$ +DECLARE + r record; + r2 record; + cond text; + idx_ctids tid[]; + ss_ctids tid[]; + count int; + plan_ok bool; + plan_line text; +BEGIN + FOR r IN SELECT colname, oper, typ, value[ordinality], matches[ordinality] FROM brinopers_bloom, unnest(op) WITH ORDINALITY AS oper LOOP + + -- prepare the condition + IF r.value IS NULL THEN + cond := format('%I %s %L', r.colname, r.oper, r.value); + ELSE + cond := format('%I %s %L::%s', r.colname, r.oper, r.value, r.typ); + END IF; + + -- run the query using the brin index + SET enable_seqscan = 0; + SET enable_bitmapscan = 1; + + plan_ok := false; + FOR plan_line IN EXECUTE format($y$EXPLAIN SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond) LOOP + IF plan_line LIKE '%Bitmap Heap Scan on brintest_bloom%' THEN + plan_ok := true; + END IF; + END LOOP; + IF NOT plan_ok THEN + RAISE WARNING 'did not get bitmap indexscan plan for %', r; + END IF; + + EXECUTE format($y$SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond) + INTO idx_ctids; + + -- run the query using a seqscan + SET enable_seqscan = 1; + SET enable_bitmapscan = 0; + + plan_ok := false; + FOR plan_line IN EXECUTE format($y$EXPLAIN SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond) LOOP + IF plan_line LIKE '%Seq Scan on brintest_bloom%' THEN + plan_ok := true; + END IF; + END LOOP; + IF NOT plan_ok THEN + RAISE WARNING 'did not get seqscan plan for %', r; + END IF; + + EXECUTE format($y$SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond) + INTO ss_ctids; + + -- make sure both return the same results + count := array_length(idx_ctids, 1); + + IF NOT (count = array_length(ss_ctids, 1) AND + idx_ctids @> ss_ctids AND + idx_ctids <@ ss_ctids) THEN + -- report the results of each scan to make the differences obvious + RAISE WARNING 'something not right in %: count %', r, count; + SET enable_seqscan = 1; + SET enable_bitmapscan = 0; + FOR r2 IN EXECUTE 'SELECT ' || r.colname || ' FROM brintest_bloom WHERE ' || cond LOOP + RAISE NOTICE 'seqscan: %', r2; + END LOOP; + + SET enable_seqscan = 0; + SET enable_bitmapscan = 1; + FOR r2 IN EXECUTE 'SELECT ' || r.colname || ' FROM brintest_bloom WHERE ' || cond LOOP + RAISE NOTICE 'bitmapscan: %', r2; + END LOOP; + END IF; + + -- make sure we found expected number of matches + IF count != r.matches THEN RAISE WARNING 'unexpected number of results % for %', count, r; END IF; + END LOOP; +END; +$x$; +RESET enable_seqscan; +RESET enable_bitmapscan; +INSERT INTO brintest_bloom SELECT + repeat(stringu1, 42)::bytea, + substr(stringu1, 1, 1)::"char", + stringu1::name, 142857 * tenthous, + thousand, + twothousand, + repeat(stringu1, 42), + unique1::oid, + (four + 1.0)/(hundred+1), + odd::float8 / (tenthous + 1), + format('%s:00:%s:00:%s:00', to_hex(odd), to_hex(even), to_hex(hundred))::macaddr, + inet '10.2.3.4' + tenthous, + cidr '10.2.3/24' + tenthous, + substr(stringu1, 1, 1)::bpchar, + date '1995-08-15' + tenthous, + time '01:20:30' + thousand * interval '18.5 second', + timestamp '1942-07-23 03:05:09' + tenthous * interval '36.38 hours', + timestamptz '1972-10-10 03:00' + thousand * interval '1 hour', + justify_days(justify_hours(tenthous * interval '12 minutes')), + timetz '01:30:20' + hundred * interval '15 seconds', + tenthous::numeric(36,30) * fivethous * even / (hundred + 1), + format('%s%s-%s-%s-%s-%s%s%s', to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'))::uuid, + format('%s/%s%s', odd, even, tenthous)::pg_lsn +FROM tenk1 ORDER BY unique2 LIMIT 5 OFFSET 5; +SELECT brin_desummarize_range('brinidx_bloom', 0); + brin_desummarize_range +------------------------ + +(1 row) + +VACUUM brintest_bloom; -- force a summarization cycle in brinidx +UPDATE brintest_bloom SET int8col = int8col * int4col; +UPDATE brintest_bloom SET textcol = '' WHERE textcol IS NOT NULL; +-- Tests for brin_summarize_new_values +SELECT brin_summarize_new_values('brintest_bloom'); -- error, not an index +ERROR: "brintest_bloom" is not an index +SELECT brin_summarize_new_values('tenk1_unique1'); -- error, not a BRIN index +ERROR: "tenk1_unique1" is not a BRIN index +SELECT brin_summarize_new_values('brinidx_bloom'); -- ok, no change expected + brin_summarize_new_values +--------------------------- + 0 +(1 row) + +-- Tests for brin_desummarize_range +SELECT brin_desummarize_range('brinidx_bloom', -1); -- error, invalid range +ERROR: block number out of range: -1 +SELECT brin_desummarize_range('brinidx_bloom', 0); + brin_desummarize_range +------------------------ + +(1 row) + +SELECT brin_desummarize_range('brinidx_bloom', 0); + brin_desummarize_range +------------------------ + +(1 row) + +SELECT brin_desummarize_range('brinidx_bloom', 100000000); + brin_desummarize_range +------------------------ + +(1 row) + +-- Test brin_summarize_range +CREATE TABLE brin_summarize_bloom ( + value int +) WITH (fillfactor=10, autovacuum_enabled=false); +CREATE INDEX brin_summarize_bloom_idx ON brin_summarize_bloom USING brin (value) WITH (pages_per_range=2); +-- Fill a few pages +DO $$ +DECLARE curtid tid; +BEGIN + LOOP + INSERT INTO brin_summarize_bloom VALUES (1) RETURNING ctid INTO curtid; + EXIT WHEN curtid > tid '(2, 0)'; + END LOOP; +END; +$$; +-- summarize one range +SELECT brin_summarize_range('brin_summarize_bloom_idx', 0); + brin_summarize_range +---------------------- + 0 +(1 row) + +-- nothing: already summarized +SELECT brin_summarize_range('brin_summarize_bloom_idx', 1); + brin_summarize_range +---------------------- + 0 +(1 row) + +-- summarize one range +SELECT brin_summarize_range('brin_summarize_bloom_idx', 2); + brin_summarize_range +---------------------- + 1 +(1 row) + +-- nothing: page doesn't exist in table +SELECT brin_summarize_range('brin_summarize_bloom_idx', 4294967295); + brin_summarize_range +---------------------- + 0 +(1 row) + +-- invalid block number values +SELECT brin_summarize_range('brin_summarize_bloom_idx', -1); +ERROR: block number out of range: -1 +SELECT brin_summarize_range('brin_summarize_bloom_idx', 4294967296); +ERROR: block number out of range: 4294967296 +-- test brin cost estimates behave sanely based on correlation of values +CREATE TABLE brin_test_bloom (a INT, b INT); +INSERT INTO brin_test_bloom SELECT x/100,x%100 FROM generate_series(1,10000) x(x); +CREATE INDEX brin_test_bloom_a_idx ON brin_test_bloom USING brin (a) WITH (pages_per_range = 2); +CREATE INDEX brin_test_bloom_b_idx ON brin_test_bloom USING brin (b) WITH (pages_per_range = 2); +VACUUM ANALYZE brin_test_bloom; +-- Ensure brin index is used when columns are perfectly correlated +EXPLAIN (COSTS OFF) SELECT * FROM brin_test_bloom WHERE a = 1; + QUERY PLAN +-------------------------------------------------- + Bitmap Heap Scan on brin_test_bloom + Recheck Cond: (a = 1) + -> Bitmap Index Scan on brin_test_bloom_a_idx + Index Cond: (a = 1) +(4 rows) + +-- Ensure brin index is not used when values are not correlated +EXPLAIN (COSTS OFF) SELECT * FROM brin_test_bloom WHERE b = 1; + QUERY PLAN +----------------------------- + Seq Scan on brin_test_bloom + Filter: (b = 1) +(2 rows) + diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 85af36ee5b..400f333ef2 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1924,6 +1924,7 @@ ORDER BY 1, 2, 3; 2742 | 16 | @@ 3580 | 1 | < 3580 | 1 | << + 3580 | 1 | = 3580 | 2 | &< 3580 | 2 | <= 3580 | 3 | && @@ -1987,7 +1988,7 @@ ORDER BY 1, 2, 3; 4000 | 26 | >> 4000 | 27 | >>= 4000 | 28 | ^@ -(125 rows) +(126 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule index f23fe8d870..38a18660da 100644 --- a/src/test/regress/parallel_schedule +++ b/src/test/regress/parallel_schedule @@ -75,6 +75,11 @@ test: select_into select_distinct select_distinct_on select_implicit select_havi # ---------- test: brin gin gist spgist privileges init_privs security_label collate matview lock replica_identity rowsecurity object_address tablesample groupingsets drop_operator password identity generated join_hash +# ---------- +# Additional BRIN tests +# ---------- +test: brin_bloom + # ---------- # Another group of parallel tests # ---------- diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule index ca200eb599..56fb53d053 100644 --- a/src/test/regress/serial_schedule +++ b/src/test/regress/serial_schedule @@ -105,6 +105,7 @@ test: delete test: namespace test: prepared_xacts test: brin +test: brin_bloom test: gin test: gist test: spgist diff --git a/src/test/regress/sql/brin_bloom.sql b/src/test/regress/sql/brin_bloom.sql new file mode 100644 index 0000000000..abd5a33deb --- /dev/null +++ b/src/test/regress/sql/brin_bloom.sql @@ -0,0 +1,404 @@ +CREATE TABLE brintest_bloom (byteacol bytea, + charcol "char", + namecol name, + int8col bigint, + int2col smallint, + int4col integer, + textcol text, + oidcol oid, + float4col real, + float8col double precision, + macaddrcol macaddr, + inetcol inet, + cidrcol cidr, + bpcharcol character, + datecol date, + timecol time without time zone, + timestampcol timestamp without time zone, + timestamptzcol timestamp with time zone, + intervalcol interval, + timetzcol time with time zone, + numericcol numeric, + uuidcol uuid, + lsncol pg_lsn +) WITH (fillfactor=10); + +INSERT INTO brintest_bloom SELECT + repeat(stringu1, 8)::bytea, + substr(stringu1, 1, 1)::"char", + stringu1::name, 142857 * tenthous, + thousand, + twothousand, + repeat(stringu1, 8), + unique1::oid, + (four + 1.0)/(hundred+1), + odd::float8 / (tenthous + 1), + format('%s:00:%s:00:%s:00', to_hex(odd), to_hex(even), to_hex(hundred))::macaddr, + inet '10.2.3.4/24' + tenthous, + cidr '10.2.3/24' + tenthous, + substr(stringu1, 1, 1)::bpchar, + date '1995-08-15' + tenthous, + time '01:20:30' + thousand * interval '18.5 second', + timestamp '1942-07-23 03:05:09' + tenthous * interval '36.38 hours', + timestamptz '1972-10-10 03:00' + thousand * interval '1 hour', + justify_days(justify_hours(tenthous * interval '12 minutes')), + timetz '01:30:20+02' + hundred * interval '15 seconds', + tenthous::numeric(36,30) * fivethous * even / (hundred + 1), + format('%s%s-%s-%s-%s-%s%s%s', to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'))::uuid, + format('%s/%s%s', odd, even, tenthous)::pg_lsn +FROM tenk1 ORDER BY unique2 LIMIT 100; + +-- throw in some NULL's and different values +INSERT INTO brintest_bloom (inetcol, cidrcol) SELECT + inet 'fe80::6e40:8ff:fea9:8c46' + tenthous, + cidr 'fe80::6e40:8ff:fea9:8c46' + tenthous +FROM tenk1 ORDER BY thousand, tenthous LIMIT 25; + +-- test bloom specific index options +-- ndistinct must be >= -1.0 +CREATE INDEX brinidx_bloom ON brintest_bloom USING brin ( + byteacol bytea_bloom_ops(n_distinct_per_range = -1.1) +); +-- false_positive_rate must be between 0.001 and 1.0 +CREATE INDEX brinidx_bloom ON brintest_bloom USING brin ( + byteacol bytea_bloom_ops(false_positive_rate = 0.0) +); +CREATE INDEX brinidx_bloom ON brintest_bloom USING brin ( + byteacol bytea_bloom_ops(false_positive_rate = 1.01) +); + +CREATE INDEX brinidx_bloom ON brintest_bloom USING brin ( + byteacol bytea_bloom_ops, + charcol char_bloom_ops, + namecol name_bloom_ops, + int8col int8_bloom_ops, + int2col int2_bloom_ops, + int4col int4_bloom_ops, + textcol text_bloom_ops, + oidcol oid_bloom_ops, + float4col float4_bloom_ops, + float8col float8_bloom_ops, + macaddrcol macaddr_bloom_ops, + inetcol inet_bloom_ops, + cidrcol inet_bloom_ops, + bpcharcol bpchar_bloom_ops, + datecol date_bloom_ops, + timecol time_bloom_ops, + timestampcol timestamp_bloom_ops, + timestamptzcol timestamptz_bloom_ops, + intervalcol interval_bloom_ops, + timetzcol timetz_bloom_ops, + numericcol numeric_bloom_ops, + uuidcol uuid_bloom_ops, + lsncol pg_lsn_bloom_ops +) with (pages_per_range = 1); + +CREATE TABLE brinopers_bloom (colname name, typ text, + op text[], value text[], matches int[], + check (cardinality(op) = cardinality(value)), + check (cardinality(op) = cardinality(matches))); + +INSERT INTO brinopers_bloom VALUES + ('byteacol', 'bytea', + '{=}', + '{BNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAA}', + '{1}'), + ('charcol', '"char"', + '{=}', + '{M}', + '{6}'), + ('namecol', 'name', + '{=}', + '{MAAAAA}', + '{2}'), + ('int2col', 'int2', + '{=}', + '{800}', + '{1}'), + ('int2col', 'int4', + '{=}', + '{800}', + '{1}'), + ('int2col', 'int8', + '{=}', + '{800}', + '{1}'), + ('int4col', 'int2', + '{=}', + '{800}', + '{1}'), + ('int4col', 'int4', + '{=}', + '{800}', + '{1}'), + ('int4col', 'int8', + '{=}', + '{800}', + '{1}'), + ('int8col', 'int8', + '{=}', + '{1257141600}', + '{1}'), + ('textcol', 'text', + '{=}', + '{BNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAABNAAAA}', + '{1}'), + ('oidcol', 'oid', + '{=}', + '{8800}', + '{1}'), + ('float4col', 'float4', + '{=}', + '{1}', + '{4}'), + ('float4col', 'float8', + '{=}', + '{1}', + '{4}'), + ('float8col', 'float4', + '{=}', + '{0}', + '{1}'), + ('float8col', 'float8', + '{=}', + '{0}', + '{1}'), + ('macaddrcol', 'macaddr', + '{=}', + '{2c:00:2d:00:16:00}', + '{2}'), + ('inetcol', 'inet', + '{=}', + '{10.2.14.231/24}', + '{1}'), + ('inetcol', 'cidr', + '{=}', + '{fe80::6e40:8ff:fea9:8c46}', + '{1}'), + ('cidrcol', 'inet', + '{=}', + '{10.2.14/24}', + '{2}'), + ('cidrcol', 'inet', + '{=}', + '{fe80::6e40:8ff:fea9:8c46}', + '{1}'), + ('cidrcol', 'cidr', + '{=}', + '{10.2.14/24}', + '{2}'), + ('cidrcol', 'cidr', + '{=}', + '{fe80::6e40:8ff:fea9:8c46}', + '{1}'), + ('bpcharcol', 'bpchar', + '{=}', + '{W}', + '{6}'), + ('datecol', 'date', + '{=}', + '{2009-12-01}', + '{1}'), + ('timecol', 'time', + '{=}', + '{02:28:57}', + '{1}'), + ('timestampcol', 'timestamp', + '{=}', + '{1964-03-24 19:26:45}', + '{1}'), + ('timestampcol', 'timestamptz', + '{=}', + '{1964-03-24 19:26:45}', + '{1}'), + ('timestamptzcol', 'timestamptz', + '{=}', + '{1972-10-19 09:00:00-07}', + '{1}'), + ('intervalcol', 'interval', + '{=}', + '{1 mons 13 days 12:24}', + '{1}'), + ('timetzcol', 'timetz', + '{=}', + '{01:35:50+02}', + '{2}'), + ('numericcol', 'numeric', + '{=}', + '{2268164.347826086956521739130434782609}', + '{1}'), + ('uuidcol', 'uuid', + '{=}', + '{52225222-5222-5222-5222-522252225222}', + '{1}'), + ('lsncol', 'pg_lsn', + '{=, IS, IS NOT}', + '{44/455222, NULL, NULL}', + '{1, 25, 100}'); + +DO $x$ +DECLARE + r record; + r2 record; + cond text; + idx_ctids tid[]; + ss_ctids tid[]; + count int; + plan_ok bool; + plan_line text; +BEGIN + FOR r IN SELECT colname, oper, typ, value[ordinality], matches[ordinality] FROM brinopers_bloom, unnest(op) WITH ORDINALITY AS oper LOOP + + -- prepare the condition + IF r.value IS NULL THEN + cond := format('%I %s %L', r.colname, r.oper, r.value); + ELSE + cond := format('%I %s %L::%s', r.colname, r.oper, r.value, r.typ); + END IF; + + -- run the query using the brin index + SET enable_seqscan = 0; + SET enable_bitmapscan = 1; + + plan_ok := false; + FOR plan_line IN EXECUTE format($y$EXPLAIN SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond) LOOP + IF plan_line LIKE '%Bitmap Heap Scan on brintest_bloom%' THEN + plan_ok := true; + END IF; + END LOOP; + IF NOT plan_ok THEN + RAISE WARNING 'did not get bitmap indexscan plan for %', r; + END IF; + + EXECUTE format($y$SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond) + INTO idx_ctids; + + -- run the query using a seqscan + SET enable_seqscan = 1; + SET enable_bitmapscan = 0; + + plan_ok := false; + FOR plan_line IN EXECUTE format($y$EXPLAIN SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond) LOOP + IF plan_line LIKE '%Seq Scan on brintest_bloom%' THEN + plan_ok := true; + END IF; + END LOOP; + IF NOT plan_ok THEN + RAISE WARNING 'did not get seqscan plan for %', r; + END IF; + + EXECUTE format($y$SELECT array_agg(ctid) FROM brintest_bloom WHERE %s $y$, cond) + INTO ss_ctids; + + -- make sure both return the same results + count := array_length(idx_ctids, 1); + + IF NOT (count = array_length(ss_ctids, 1) AND + idx_ctids @> ss_ctids AND + idx_ctids <@ ss_ctids) THEN + -- report the results of each scan to make the differences obvious + RAISE WARNING 'something not right in %: count %', r, count; + SET enable_seqscan = 1; + SET enable_bitmapscan = 0; + FOR r2 IN EXECUTE 'SELECT ' || r.colname || ' FROM brintest_bloom WHERE ' || cond LOOP + RAISE NOTICE 'seqscan: %', r2; + END LOOP; + + SET enable_seqscan = 0; + SET enable_bitmapscan = 1; + FOR r2 IN EXECUTE 'SELECT ' || r.colname || ' FROM brintest_bloom WHERE ' || cond LOOP + RAISE NOTICE 'bitmapscan: %', r2; + END LOOP; + END IF; + + -- make sure we found expected number of matches + IF count != r.matches THEN RAISE WARNING 'unexpected number of results % for %', count, r; END IF; + END LOOP; +END; +$x$; + +RESET enable_seqscan; +RESET enable_bitmapscan; + +INSERT INTO brintest_bloom SELECT + repeat(stringu1, 42)::bytea, + substr(stringu1, 1, 1)::"char", + stringu1::name, 142857 * tenthous, + thousand, + twothousand, + repeat(stringu1, 42), + unique1::oid, + (four + 1.0)/(hundred+1), + odd::float8 / (tenthous + 1), + format('%s:00:%s:00:%s:00', to_hex(odd), to_hex(even), to_hex(hundred))::macaddr, + inet '10.2.3.4' + tenthous, + cidr '10.2.3/24' + tenthous, + substr(stringu1, 1, 1)::bpchar, + date '1995-08-15' + tenthous, + time '01:20:30' + thousand * interval '18.5 second', + timestamp '1942-07-23 03:05:09' + tenthous * interval '36.38 hours', + timestamptz '1972-10-10 03:00' + thousand * interval '1 hour', + justify_days(justify_hours(tenthous * interval '12 minutes')), + timetz '01:30:20' + hundred * interval '15 seconds', + tenthous::numeric(36,30) * fivethous * even / (hundred + 1), + format('%s%s-%s-%s-%s-%s%s%s', to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'), to_char(tenthous, 'FM0000'))::uuid, + format('%s/%s%s', odd, even, tenthous)::pg_lsn +FROM tenk1 ORDER BY unique2 LIMIT 5 OFFSET 5; + +SELECT brin_desummarize_range('brinidx_bloom', 0); +VACUUM brintest_bloom; -- force a summarization cycle in brinidx + +UPDATE brintest_bloom SET int8col = int8col * int4col; +UPDATE brintest_bloom SET textcol = '' WHERE textcol IS NOT NULL; + +-- Tests for brin_summarize_new_values +SELECT brin_summarize_new_values('brintest_bloom'); -- error, not an index +SELECT brin_summarize_new_values('tenk1_unique1'); -- error, not a BRIN index +SELECT brin_summarize_new_values('brinidx_bloom'); -- ok, no change expected + +-- Tests for brin_desummarize_range +SELECT brin_desummarize_range('brinidx_bloom', -1); -- error, invalid range +SELECT brin_desummarize_range('brinidx_bloom', 0); +SELECT brin_desummarize_range('brinidx_bloom', 0); +SELECT brin_desummarize_range('brinidx_bloom', 100000000); + +-- Test brin_summarize_range +CREATE TABLE brin_summarize_bloom ( + value int +) WITH (fillfactor=10, autovacuum_enabled=false); +CREATE INDEX brin_summarize_bloom_idx ON brin_summarize_bloom USING brin (value) WITH (pages_per_range=2); +-- Fill a few pages +DO $$ +DECLARE curtid tid; +BEGIN + LOOP + INSERT INTO brin_summarize_bloom VALUES (1) RETURNING ctid INTO curtid; + EXIT WHEN curtid > tid '(2, 0)'; + END LOOP; +END; +$$; + +-- summarize one range +SELECT brin_summarize_range('brin_summarize_bloom_idx', 0); +-- nothing: already summarized +SELECT brin_summarize_range('brin_summarize_bloom_idx', 1); +-- summarize one range +SELECT brin_summarize_range('brin_summarize_bloom_idx', 2); +-- nothing: page doesn't exist in table +SELECT brin_summarize_range('brin_summarize_bloom_idx', 4294967295); +-- invalid block number values +SELECT brin_summarize_range('brin_summarize_bloom_idx', -1); +SELECT brin_summarize_range('brin_summarize_bloom_idx', 4294967296); + + +-- test brin cost estimates behave sanely based on correlation of values +CREATE TABLE brin_test_bloom (a INT, b INT); +INSERT INTO brin_test_bloom SELECT x/100,x%100 FROM generate_series(1,10000) x(x); +CREATE INDEX brin_test_bloom_a_idx ON brin_test_bloom USING brin (a) WITH (pages_per_range = 2); +CREATE INDEX brin_test_bloom_b_idx ON brin_test_bloom USING brin (b) WITH (pages_per_range = 2); +VACUUM ANALYZE brin_test_bloom; + +-- Ensure brin index is used when columns are perfectly correlated +EXPLAIN (COSTS OFF) SELECT * FROM brin_test_bloom WHERE a = 1; +-- Ensure brin index is not used when values are not correlated +EXPLAIN (COSTS OFF) SELECT * FROM brin_test_bloom WHERE b = 1; -- 2.20.1 --54rc27wh4iahvygf Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="0010-BRIN-multi-range-minmax-indexes-20190611.patch" ^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v6 15/15] Trigger authors need not worry about parallelism @ 2023-10-02 19:55 Karl O. Pinc <[email protected]> 0 siblings, 0 replies; 14+ messages in thread From: Karl O. Pinc @ 2023-10-02 19:55 UTC (permalink / raw) The idea is that it's worth telling the trigger author who comes across PARALLEL declarations when creating trigger functions that they don't need to investigate further. Is this stating the obvious? Yes. But it's not obvious until you've read about PGs parallelism, and read _enough_ to know that there's only mention of queries and no mention of data modification statements. Knowing that something is missing from PG requires at least passing familiarity with _all_ of the documentation. Plus, this patch adds an index entry so the new verbage is easy to find for those who do investigate. Agreed, documenting everything PG does _not_ do is a terrible idea. The goal here is to help trigger authors limit what they need to research. --- doc/src/sgml/trigger.sgml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/doc/src/sgml/trigger.sgml b/doc/src/sgml/trigger.sgml index 6e1f370b21..f6ec8d2a97 100644 --- a/doc/src/sgml/trigger.sgml +++ b/doc/src/sgml/trigger.sgml @@ -377,6 +377,17 @@ trigger. </para> + <indexterm> + <primary>parallel safety</primary> + <secondary>triggers</secondary> + </indexterm> + + <para> + Trigger authors need not concern themselves with parallel safety. + <link linkend="parallel-query">Parallel execution</link> + is supported only in queries, + </para> + <para> Each programming language that supports triggers has its own method for making the trigger input data available to the trigger function. -- 2.30.2 --MP_/74urtnrsBSymuH7bJczNOGS-- ^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v7 15/16] Trigger authors need not worry about parallelism @ 2023-10-02 19:55 Karl O. Pinc <[email protected]> 0 siblings, 0 replies; 14+ messages in thread From: Karl O. Pinc @ 2023-10-02 19:55 UTC (permalink / raw) The idea is that it's worth telling the trigger author who comes across PARALLEL declarations when creating trigger functions that they don't need to investigate further. Is this stating the obvious? Yes. But it's not obvious until you've read about PGs parallelism, and read _enough_ to know that there's only mention of queries and no mention of data modification statements. Knowing that something is missing from PG requires at least passing familiarity with _all_ of the documentation. Plus, this patch adds an index entry so the new verbage is easy to find for those who do investigate. Agreed, documenting everything PG does _not_ do is a terrible idea. The goal here is to help trigger authors limit what they need to research. --- doc/src/sgml/trigger.sgml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/doc/src/sgml/trigger.sgml b/doc/src/sgml/trigger.sgml index 6e1f370b21..f6ec8d2a97 100644 --- a/doc/src/sgml/trigger.sgml +++ b/doc/src/sgml/trigger.sgml @@ -377,6 +377,17 @@ trigger. </para> + <indexterm> + <primary>parallel safety</primary> + <secondary>triggers</secondary> + </indexterm> + + <para> + Trigger authors need not concern themselves with parallel safety. + <link linkend="parallel-query">Parallel execution</link> + is supported only in queries, + </para> + <para> Each programming language that supports triggers has its own method for making the trigger input data available to the trigger function. -- 2.30.2 --MP_/VSGM3xNEmY7iJyL2wuWRCjV Content-Type: text/x-patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=v7-0016-Predicate-locks-are-held-per-cluster-not-per-data.patch ^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v8 2/3] Add the pg_stat_lock view @ 2025-07-31 09:35 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 14+ messages in thread From: Bertrand Drouvot @ 2025-07-31 09:35 UTC (permalink / raw) This new view reports lock statistics. Note that it does not omit combinations which do not make sense (as pg_locks does). Also wait_time is reported as bigint as the deadlock_timeout default value is 1s. This commit also adds documentation and a few tests. XXX: Bump catversion --- doc/src/sgml/monitoring.sgml | 113 ++++++++++++++ src/backend/catalog/system_views.sql | 9 ++ src/backend/utils/activity/pgstat_lock.c | 8 + src/backend/utils/adt/pgstatfuncs.c | 39 +++++ src/include/catalog/pg_proc.dat | 9 ++ src/include/pgstat.h | 1 + src/test/isolation/expected/stats.out | 189 +++++++++++++++++++++++ src/test/isolation/expected/stats_1.out | 189 +++++++++++++++++++++++ src/test/isolation/specs/stats.spec | 95 ++++++++++++ src/test/regress/expected/rules.out | 6 + src/test/regress/expected/stats.out | 48 ++++++ src/test/regress/sql/stats.sql | 36 +++++ 12 files changed, 742 insertions(+) 15.9% doc/src/sgml/ 5.0% src/backend/utils/adt/ 54.1% src/test/isolation/expected/ 11.3% src/test/isolation/specs/ 5.7% src/test/regress/expected/ 4.0% src/test/regress/sql/ 3.6% src/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index b77d189a500..3a196bc305c 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -493,6 +493,15 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser </entry> </row> + <row> + <entry><structname>pg_stat_lock</structname><indexterm><primary>pg_stat_lock</primary></indexterm></entry> + <entry> + One row for each lock type, containing cluster-wide locks statistics. + See <link linkend="monitoring-pg-stat-lock-view"> + <structname>pg_stat_lock</structname></link> for details. + </entry> + </row> + <row> <entry><structname>pg_stat_replication_slots</structname><indexterm><primary>pg_stat_replication_slots</primary></indexterm></entry> <entry>One row per replication slot, showing statistics about the @@ -3124,6 +3133,104 @@ description | Waiting for a newly initialized WAL file to reach durable storage </sect2> + + <sect2 id="monitoring-pg-stat-lock-view"> + <title><structname>pg_stat_lock</structname></title> + + <indexterm> + <primary>pg_stat_lock</primary> + </indexterm> + + <para> + The <structname>pg_stat_lock</structname> view will contain one row for each + lock type, showing cluster-wide locks statistics. + </para> + + <table id="pg-stat-lock-view" xreflabel="pg_stat_lock"> + <title><structname>pg_stat_lock</structname> View</title> + <tgroup cols="1"> + <thead> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + Column Type + </para> + <para> + Description + </para> + </entry> + </row> + </thead> + <tbody> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>locktype</structfield> <type>text</type> + </para> + <para> + Type of the lockable object. See <link linkend="view-pg-locks"> + <structname>pg_locks</structname></link> for details. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>waits</structfield> <type>bigint</type> + </para> + <para> + Number of times a lock of this type had to wait because of a + conflicting lock. Only incremented when <xref linkend="guc-log-lock-waits"/> + is enabled and the lock was successfully acquired after waiting longer + than <xref linkend="guc-deadlock-timeout"/>. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>wait_time</structfield> <type>bigint</type> + </para> + <para> + Total time spent waiting for locks of this type, in milliseconds. + Only incremented when <xref linkend="guc-log-lock-waits"/> is enabled and + the lock was successfully acquired after waiting longer than + <xref linkend="guc-deadlock-timeout"/>. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>fastpath_exceeded</structfield> <type>bigint</type> + </para> + <para> + Number of times a lock of this type could not be acquired via fast path + because the fast path slot limit was exceeded. You may want to increase + <xref linkend="guc-max-locks-per-transaction"/> if you feel this counter + is too high. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>stats_reset</structfield> <type>timestamp with time zone</type> + </para> + <para> + Time at which these statistics were last reset. + </para> + </entry> + </row> + </tbody> + </tgroup> + </table> + </sect2> + <sect2 id="monitoring-pg-stat-bgwriter-view"> <title><structname>pg_stat_bgwriter</structname></title> @@ -5195,6 +5302,12 @@ description | Waiting for a newly initialized WAL file to reach durable storage <structname>pg_stat_io</structname> view. </para> </listitem> + <listitem> + <para> + <literal>lock</literal>: Reset all the counters shown in the + <structname>pg_stat_lock</structname> view. + </para> + </listitem> <listitem> <para> <literal>recovery_prefetch</literal>: Reset all the counters shown in diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 1ea8f1faa9e..22efeb713e4 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -985,6 +985,15 @@ CREATE VIEW pg_stat_slru AS s.stats_reset FROM pg_stat_get_slru() s; +CREATE VIEW pg_stat_lock AS + SELECT + l.locktype, + l.waits, + l.wait_time, + l.fastpath_exceeded, + l.stats_reset + FROM pg_stat_get_lock() l; + CREATE VIEW pg_stat_wal_receiver AS SELECT s.pid, diff --git a/src/backend/utils/activity/pgstat_lock.c b/src/backend/utils/activity/pgstat_lock.c index 91a53a6013a..954fa51c3e0 100644 --- a/src/backend/utils/activity/pgstat_lock.c +++ b/src/backend/utils/activity/pgstat_lock.c @@ -21,6 +21,14 @@ static PgStat_PendingLock PendingLockStats; static bool have_lockstats = false; +PgStat_Lock * +pgstat_fetch_stat_lock(void) +{ + pgstat_snapshot_fixed(PGSTAT_KIND_LOCK); + + return &pgStatLocal.snapshot.lock; +} + /* * Simpler wrapper of pgstat_lock_flush_cb() */ diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index b1df96e7b0b..27ca2d8f07c 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1737,6 +1737,42 @@ pg_stat_get_wal(PG_FUNCTION_ARGS) wal_stats->stat_reset_timestamp)); } +Datum +pg_stat_get_lock(PG_FUNCTION_ARGS) +{ +#define PG_STAT_LOCK_COLS 5 + ReturnSetInfo *rsinfo; + PgStat_Lock *lock_stats; + + InitMaterializedSRF(fcinfo, 0); + rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; + + lock_stats = pgstat_fetch_stat_lock(); + + for (int lcktype = 0; lcktype <= LOCKTAG_LAST_TYPE; lcktype++) + { + const char *locktypename; + Datum values[PG_STAT_LOCK_COLS] = {0}; + bool nulls[PG_STAT_LOCK_COLS] = {0}; + PgStat_LockEntry *lck_stats = &lock_stats->stats[lcktype]; + int i = 0; + + locktypename = LockTagTypeNames[lcktype]; + + values[i++] = CStringGetTextDatum(locktypename); + values[i++] = Int64GetDatum(lck_stats->waits); + values[i++] = Int64GetDatum(lck_stats->wait_time); + values[i++] = Int64GetDatum(lck_stats->fastpath_exceeded); + values[i] = TimestampTzGetDatum(lock_stats->stat_reset_timestamp); + + Assert(i + 1 == PG_STAT_LOCK_COLS); + + tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls); + } + + return (Datum) 0; +} + /* * Returns statistics of SLRU caches. */ @@ -1921,6 +1957,7 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) pgstat_reset_of_kind(PGSTAT_KIND_BGWRITER); pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER); pgstat_reset_of_kind(PGSTAT_KIND_IO); + pgstat_reset_of_kind(PGSTAT_KIND_LOCK); XLogPrefetchResetStats(); pgstat_reset_of_kind(PGSTAT_KIND_SLRU); pgstat_reset_of_kind(PGSTAT_KIND_WAL); @@ -1938,6 +1975,8 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER); else if (strcmp(target, "io") == 0) pgstat_reset_of_kind(PGSTAT_KIND_IO); + else if (strcmp(target, "lock") == 0) + pgstat_reset_of_kind(PGSTAT_KIND_LOCK); else if (strcmp(target, "recovery_prefetch") == 0) XLogPrefetchResetStats(); else if (strcmp(target, "slru") == 0) diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index dac40992cbc..daeb45704c6 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6027,6 +6027,15 @@ proargnames => '{backend_type,object,context,reads,read_bytes,read_time,writes,write_bytes,write_time,writebacks,writeback_time,extends,extend_bytes,extend_time,hits,evictions,reuses,fsyncs,fsync_time,stats_reset}', prosrc => 'pg_stat_get_io' }, +{ oid => '9375', descr => 'statistics: per lock type statistics', + proname => 'pg_stat_get_lock', prorows => '10', proretset => 't', + provolatile => 'v', proparallel => 'r', prorettype => 'record', + proargtypes => '', + proallargtypes => '{text,int8,int8,int8,timestamptz}', + proargmodes => '{o,o,o,o,o}', + proargnames => '{locktype,waits,wait_time,fastpath_exceeded,stats_reset}', + prosrc => 'pg_stat_get_lock' }, + { oid => '6386', descr => 'statistics: backend IO statistics', proname => 'pg_stat_get_backend_io', prorows => '5', proretset => 't', provolatile => 'v', proparallel => 'r', prorettype => 'record', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index f563132bf7b..1c261e2c7a8 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -644,6 +644,7 @@ extern bool pgstat_tracks_io_op(BackendType bktype, IOObject io_object, extern void pgstat_lock_flush(bool nowait); extern void pgstat_count_lock_fastpath_exceeded(uint8 locktag_type); extern void pgstat_count_lock_waits(uint8 locktag_type, long msecs); +extern PgStat_Lock *pgstat_fetch_stat_lock(void); /* * Functions in pgstat_database.c diff --git a/src/test/isolation/expected/stats.out b/src/test/isolation/expected/stats.out index cfad309ccf3..3cae3052e40 100644 --- a/src/test/isolation/expected/stats.out +++ b/src/test/isolation/expected/stats.out @@ -3751,3 +3751,192 @@ test_stat_func| 1|t |t (1 row) step s1_commit: COMMIT; + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_begin s1_lock_relation s2_begin s2_ff s2_lock_relation s1_sleep s1_commit s2_commit s2_report_stat_lock_relation +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_begin: BEGIN; +step s1_lock_relation: LOCK TABLE test_stat_tab; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_relation: LOCK TABLE test_stat_tab; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_lock_relation: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_relation: SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column? +--------+-------- +t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_table_insert s1_begin s1_table_update_k1 s2_begin s2_ff s2_table_update_k1 s1_sleep s1_commit s2_commit s2_report_stat_lock_transactionid +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_table_insert: INSERT INTO test_stat_tab(key, value) VALUES('k1', 1), ('k2', 1), ('k3', 1); +step s1_begin: BEGIN; +step s1_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_table_update_k1: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_transactionid: SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'transactionid'; +?column?|?column? +--------+-------- +t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_lock_advisory_lock s2_begin s2_ff s2_lock_advisory_lock s1_sleep s1_lock_advisory_unlock s2_lock_advisory_unlock s2_commit s2_report_stat_lock_advisory +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_lock_advisory_lock: SELECT pg_advisory_lock(1); +pg_advisory_lock +---------------- + +(1 row) + +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_advisory_lock: SELECT pg_advisory_lock(1); <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_lock_advisory_lock: <... completed> +pg_advisory_lock +---------------- + +(1 row) + +step s2_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_commit: COMMIT; +step s2_report_stat_lock_advisory: SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'advisory'; +?column?|?column? +--------+-------- +t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_unset_log_lock_waits s1_begin s1_lock_relation s2_begin s2_ff s2_lock_relation s1_sleep s1_commit s2_commit s2_report_stat_lock_relation +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_unset_log_lock_waits: SET log_lock_waits = off; +step s1_begin: BEGIN; +step s1_lock_relation: LOCK TABLE test_stat_tab; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_relation: LOCK TABLE test_stat_tab; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_lock_relation: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_relation: SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column? +--------+-------- +f |f +(1 row) + diff --git a/src/test/isolation/expected/stats_1.out b/src/test/isolation/expected/stats_1.out index e1d937784cb..ea4fd97a9a5 100644 --- a/src/test/isolation/expected/stats_1.out +++ b/src/test/isolation/expected/stats_1.out @@ -3775,3 +3775,192 @@ test_stat_func| 1|t |t (1 row) step s1_commit: COMMIT; + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_begin s1_lock_relation s2_begin s2_ff s2_lock_relation s1_sleep s1_commit s2_commit s2_report_stat_lock_relation +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_begin: BEGIN; +step s1_lock_relation: LOCK TABLE test_stat_tab; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_relation: LOCK TABLE test_stat_tab; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_lock_relation: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_relation: SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column? +--------+-------- +t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_table_insert s1_begin s1_table_update_k1 s2_begin s2_ff s2_table_update_k1 s1_sleep s1_commit s2_commit s2_report_stat_lock_transactionid +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_table_insert: INSERT INTO test_stat_tab(key, value) VALUES('k1', 1), ('k2', 1), ('k3', 1); +step s1_begin: BEGIN; +step s1_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_table_update_k1: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_transactionid: SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'transactionid'; +?column?|?column? +--------+-------- +t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_lock_advisory_lock s2_begin s2_ff s2_lock_advisory_lock s1_sleep s1_lock_advisory_unlock s2_lock_advisory_unlock s2_commit s2_report_stat_lock_advisory +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_lock_advisory_lock: SELECT pg_advisory_lock(1); +pg_advisory_lock +---------------- + +(1 row) + +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_advisory_lock: SELECT pg_advisory_lock(1); <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_lock_advisory_lock: <... completed> +pg_advisory_lock +---------------- + +(1 row) + +step s2_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_commit: COMMIT; +step s2_report_stat_lock_advisory: SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'advisory'; +?column?|?column? +--------+-------- +t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_unset_log_lock_waits s1_begin s1_lock_relation s2_begin s2_ff s2_lock_relation s1_sleep s1_commit s2_commit s2_report_stat_lock_relation +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_unset_log_lock_waits: SET log_lock_waits = off; +step s1_begin: BEGIN; +step s1_lock_relation: LOCK TABLE test_stat_tab; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_relation: LOCK TABLE test_stat_tab; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_lock_relation: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_relation: SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column? +--------+-------- +f |f +(1 row) + diff --git a/src/test/isolation/specs/stats.spec b/src/test/isolation/specs/stats.spec index da16710da0f..42be68c545f 100644 --- a/src/test/isolation/specs/stats.spec +++ b/src/test/isolation/specs/stats.spec @@ -130,6 +130,14 @@ step s1_slru_check_stats { WHERE before.stat = 'blks_zeroed'; } +# Lock stats steps +step s1_set_deadlock_timeout { SET deadlock_timeout = '10ms'; } +step s1_set_log_lock_waits { SET log_lock_waits = on; } +step s1_reset_stat_lock { SELECT pg_stat_reset_shared('lock'); } +step s1_sleep { SELECT pg_sleep(0.5); } +step s1_lock_relation { LOCK TABLE test_stat_tab; } +step s1_lock_advisory_lock { SELECT pg_advisory_lock(1); } +step s1_lock_advisory_unlock { SELECT pg_advisory_unlock(1); } session s2 setup { SET stats_fetch_consistency = 'none'; } @@ -164,6 +172,16 @@ step s2_big_notify { SELECT pg_notify('stats_test_use', repeat(i::text, current_setting('block_size')::int / 2)) FROM generate_series(1, 3) g(i); } +# Lock stats steps +step s2_set_deadlock_timeout { SET deadlock_timeout = '10ms'; } +step s2_set_log_lock_waits { SET log_lock_waits = on; } +step s2_unset_log_lock_waits { SET log_lock_waits = off; } +step s2_report_stat_lock_relation { SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; } +step s2_report_stat_lock_transactionid { SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'transactionid'; } +step s2_report_stat_lock_advisory { SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'advisory'; } +step s2_lock_relation { LOCK TABLE test_stat_tab; } +step s2_lock_advisory_lock { SELECT pg_advisory_lock(1); } +step s2_lock_advisory_unlock { SELECT pg_advisory_unlock(1); } ###################### # Function stats tests @@ -765,3 +783,80 @@ permutation s1_clear_snapshot s1_func_stats s1_commit + +###################### +# Lock stats tests +###################### + +# relation lock + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_set_log_lock_waits + s1_begin + s1_lock_relation + s2_begin + s2_ff + s2_lock_relation + s1_sleep + s1_commit + s2_commit + s2_report_stat_lock_relation + +# transaction lock + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_set_log_lock_waits + s1_table_insert + s1_begin + s1_table_update_k1 + s2_begin + s2_ff + s2_table_update_k1 + s1_sleep + s1_commit + s2_commit + s2_report_stat_lock_transactionid + +# advisory lock + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_set_log_lock_waits + s1_lock_advisory_lock + s2_begin + s2_ff + s2_lock_advisory_lock + s1_sleep + s1_lock_advisory_unlock + s2_lock_advisory_unlock + s2_commit + s2_report_stat_lock_advisory + +# Ensure log_lock_waits behaves correctly + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_unset_log_lock_waits + s1_begin + s1_lock_relation + s2_begin + s2_ff + s2_lock_relation + s1_sleep + s1_commit + s2_commit + s2_report_stat_lock_relation diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index 78a37d9fc8f..c013fbebb88 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1951,6 +1951,12 @@ pg_stat_io| SELECT backend_type, fsync_time, stats_reset FROM pg_stat_get_io() b(backend_type, object, context, reads, read_bytes, read_time, writes, write_bytes, write_time, writebacks, writeback_time, extends, extend_bytes, extend_time, hits, evictions, reuses, fsyncs, fsync_time, stats_reset); +pg_stat_lock| SELECT locktype, + waits, + wait_time, + fastpath_exceeded, + stats_reset + FROM pg_stat_get_lock() l(locktype, waits, wait_time, fastpath_exceeded, stats_reset); pg_stat_progress_analyze| SELECT s.pid, s.datid, d.datname, diff --git a/src/test/regress/expected/stats.out b/src/test/regress/expected/stats.out index cd00f35bf7a..93de87747d4 100644 --- a/src/test/regress/expected/stats.out +++ b/src/test/regress/expected/stats.out @@ -1910,4 +1910,52 @@ SELECT * FROM check_estimated_rows('SELECT * FROM table_fillfactor'); (1 row) DROP TABLE table_fillfactor; +-- Test fastpath_exceeded stat +CREATE TABLE part_test (id int) PARTITION BY RANGE (id); +SELECT pg_stat_reset_shared('lock'); + pg_stat_reset_shared +---------------------- + +(1 row) + +-- Create partitions (exceeds number of slots) +DO $$ +DECLARE + max_locks int; +BEGIN + SELECT setting::int INTO max_locks + FROM pg_settings + WHERE name = 'max_locks_per_transaction'; + + FOR i IN 1..(max_locks + 10) LOOP + EXECUTE format( + 'CREATE TABLE part_test_%s PARTITION OF part_test + FOR VALUES FROM (%s) TO (%s)', + i, (i-1)*1000, i*1000 + ); + END LOOP; +END; +$$; +SELECT fastpath_exceeded AS fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation' \gset +-- Needs a lock on each partition +SELECT count(*) FROM part_test; + count +------- + 0 +(1 row) + +-- Ensure pending stats are flushed +SELECT pg_stat_force_next_flush(); + pg_stat_force_next_flush +-------------------------- + +(1 row) + +SELECT fastpath_exceeded > :fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation'; + ?column? +---------- + t +(1 row) + +DROP TABLE part_test; -- End of Stats Test diff --git a/src/test/regress/sql/stats.sql b/src/test/regress/sql/stats.sql index 8768e0f27fd..db9948eedb8 100644 --- a/src/test/regress/sql/stats.sql +++ b/src/test/regress/sql/stats.sql @@ -944,4 +944,40 @@ SELECT * FROM check_estimated_rows('SELECT * FROM table_fillfactor'); DROP TABLE table_fillfactor; +-- Test fastpath_exceeded stat +CREATE TABLE part_test (id int) PARTITION BY RANGE (id); + +SELECT pg_stat_reset_shared('lock'); + +-- Create partitions (exceeds number of slots) +DO $$ +DECLARE + max_locks int; +BEGIN + SELECT setting::int INTO max_locks + FROM pg_settings + WHERE name = 'max_locks_per_transaction'; + + FOR i IN 1..(max_locks + 10) LOOP + EXECUTE format( + 'CREATE TABLE part_test_%s PARTITION OF part_test + FOR VALUES FROM (%s) TO (%s)', + i, (i-1)*1000, i*1000 + ); + END LOOP; +END; +$$; + +SELECT fastpath_exceeded AS fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation' \gset + +-- Needs a lock on each partition +SELECT count(*) FROM part_test; + +-- Ensure pending stats are flushed +SELECT pg_stat_force_next_flush(); + +SELECT fastpath_exceeded > :fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation'; + +DROP TABLE part_test; + -- End of Stats Test -- 2.34.1 --CwwtKyeZcJiDzEBE Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v8-0003-Introduce-a-new-track_lock_timing-GUC.patch" ^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v5 2/2] Add the pg_stat_lock view @ 2025-07-31 09:35 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 14+ messages in thread From: Bertrand Drouvot @ 2025-07-31 09:35 UTC (permalink / raw) This new view reports lock statistics. Note that it does not omit combinations which do not make sense (as pg_locks does). Also wait_time is reported as bigint as the deadlock_timeout default value is 1s. This commit also adds documentation and a few tests. XXX: Bump catversion --- doc/src/sgml/monitoring.sgml | 113 +++++++++++++++++ src/backend/catalog/system_views.sql | 9 ++ src/backend/utils/activity/pgstat_lock.c | 8 ++ src/backend/utils/adt/pgstatfuncs.c | 39 ++++++ src/include/catalog/pg_proc.dat | 9 ++ src/include/pgstat.h | 1 + src/test/isolation/expected/stats.out | 147 +++++++++++++++++++++++ src/test/isolation/expected/stats_1.out | 147 +++++++++++++++++++++++ src/test/isolation/specs/stats.spec | 76 ++++++++++++ src/test/regress/expected/rules.out | 6 + src/test/regress/expected/stats.out | 42 +++++++ src/test/regress/sql/stats.sql | 30 +++++ 12 files changed, 627 insertions(+) 18.8% doc/src/sgml/ 5.9% src/backend/utils/adt/ 49.4% src/test/isolation/expected/ 11.3% src/test/isolation/specs/ 6.0% src/test/regress/expected/ 4.1% src/test/regress/sql/ 4.3% src/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index b77d189a500..3a196bc305c 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -493,6 +493,15 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser </entry> </row> + <row> + <entry><structname>pg_stat_lock</structname><indexterm><primary>pg_stat_lock</primary></indexterm></entry> + <entry> + One row for each lock type, containing cluster-wide locks statistics. + See <link linkend="monitoring-pg-stat-lock-view"> + <structname>pg_stat_lock</structname></link> for details. + </entry> + </row> + <row> <entry><structname>pg_stat_replication_slots</structname><indexterm><primary>pg_stat_replication_slots</primary></indexterm></entry> <entry>One row per replication slot, showing statistics about the @@ -3124,6 +3133,104 @@ description | Waiting for a newly initialized WAL file to reach durable storage </sect2> + + <sect2 id="monitoring-pg-stat-lock-view"> + <title><structname>pg_stat_lock</structname></title> + + <indexterm> + <primary>pg_stat_lock</primary> + </indexterm> + + <para> + The <structname>pg_stat_lock</structname> view will contain one row for each + lock type, showing cluster-wide locks statistics. + </para> + + <table id="pg-stat-lock-view" xreflabel="pg_stat_lock"> + <title><structname>pg_stat_lock</structname> View</title> + <tgroup cols="1"> + <thead> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + Column Type + </para> + <para> + Description + </para> + </entry> + </row> + </thead> + <tbody> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>locktype</structfield> <type>text</type> + </para> + <para> + Type of the lockable object. See <link linkend="view-pg-locks"> + <structname>pg_locks</structname></link> for details. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>waits</structfield> <type>bigint</type> + </para> + <para> + Number of times a lock of this type had to wait because of a + conflicting lock. Only incremented when <xref linkend="guc-log-lock-waits"/> + is enabled and the lock was successfully acquired after waiting longer + than <xref linkend="guc-deadlock-timeout"/>. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>wait_time</structfield> <type>bigint</type> + </para> + <para> + Total time spent waiting for locks of this type, in milliseconds. + Only incremented when <xref linkend="guc-log-lock-waits"/> is enabled and + the lock was successfully acquired after waiting longer than + <xref linkend="guc-deadlock-timeout"/>. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>fastpath_exceeded</structfield> <type>bigint</type> + </para> + <para> + Number of times a lock of this type could not be acquired via fast path + because the fast path slot limit was exceeded. You may want to increase + <xref linkend="guc-max-locks-per-transaction"/> if you feel this counter + is too high. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>stats_reset</structfield> <type>timestamp with time zone</type> + </para> + <para> + Time at which these statistics were last reset. + </para> + </entry> + </row> + </tbody> + </tgroup> + </table> + </sect2> + <sect2 id="monitoring-pg-stat-bgwriter-view"> <title><structname>pg_stat_bgwriter</structname></title> @@ -5195,6 +5302,12 @@ description | Waiting for a newly initialized WAL file to reach durable storage <structname>pg_stat_io</structname> view. </para> </listitem> + <listitem> + <para> + <literal>lock</literal>: Reset all the counters shown in the + <structname>pg_stat_lock</structname> view. + </para> + </listitem> <listitem> <para> <literal>recovery_prefetch</literal>: Reset all the counters shown in diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 1ea8f1faa9e..22efeb713e4 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -985,6 +985,15 @@ CREATE VIEW pg_stat_slru AS s.stats_reset FROM pg_stat_get_slru() s; +CREATE VIEW pg_stat_lock AS + SELECT + l.locktype, + l.waits, + l.wait_time, + l.fastpath_exceeded, + l.stats_reset + FROM pg_stat_get_lock() l; + CREATE VIEW pg_stat_wal_receiver AS SELECT s.pid, diff --git a/src/backend/utils/activity/pgstat_lock.c b/src/backend/utils/activity/pgstat_lock.c index 7af4bc6d018..cd4399b5ef6 100644 --- a/src/backend/utils/activity/pgstat_lock.c +++ b/src/backend/utils/activity/pgstat_lock.c @@ -21,6 +21,14 @@ static PgStat_PendingLock PendingLockStats; static bool have_lockstats = false; +PgStat_Lock * +pgstat_fetch_stat_lock(void) +{ + pgstat_snapshot_fixed(PGSTAT_KIND_LOCK); + + return &pgStatLocal.snapshot.lock; +} + /* * Simpler wrapper of pgstat_lock_flush_cb() */ diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index b1df96e7b0b..27ca2d8f07c 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1737,6 +1737,42 @@ pg_stat_get_wal(PG_FUNCTION_ARGS) wal_stats->stat_reset_timestamp)); } +Datum +pg_stat_get_lock(PG_FUNCTION_ARGS) +{ +#define PG_STAT_LOCK_COLS 5 + ReturnSetInfo *rsinfo; + PgStat_Lock *lock_stats; + + InitMaterializedSRF(fcinfo, 0); + rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; + + lock_stats = pgstat_fetch_stat_lock(); + + for (int lcktype = 0; lcktype <= LOCKTAG_LAST_TYPE; lcktype++) + { + const char *locktypename; + Datum values[PG_STAT_LOCK_COLS] = {0}; + bool nulls[PG_STAT_LOCK_COLS] = {0}; + PgStat_LockEntry *lck_stats = &lock_stats->stats[lcktype]; + int i = 0; + + locktypename = LockTagTypeNames[lcktype]; + + values[i++] = CStringGetTextDatum(locktypename); + values[i++] = Int64GetDatum(lck_stats->waits); + values[i++] = Int64GetDatum(lck_stats->wait_time); + values[i++] = Int64GetDatum(lck_stats->fastpath_exceeded); + values[i] = TimestampTzGetDatum(lock_stats->stat_reset_timestamp); + + Assert(i + 1 == PG_STAT_LOCK_COLS); + + tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls); + } + + return (Datum) 0; +} + /* * Returns statistics of SLRU caches. */ @@ -1921,6 +1957,7 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) pgstat_reset_of_kind(PGSTAT_KIND_BGWRITER); pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER); pgstat_reset_of_kind(PGSTAT_KIND_IO); + pgstat_reset_of_kind(PGSTAT_KIND_LOCK); XLogPrefetchResetStats(); pgstat_reset_of_kind(PGSTAT_KIND_SLRU); pgstat_reset_of_kind(PGSTAT_KIND_WAL); @@ -1938,6 +1975,8 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER); else if (strcmp(target, "io") == 0) pgstat_reset_of_kind(PGSTAT_KIND_IO); + else if (strcmp(target, "lock") == 0) + pgstat_reset_of_kind(PGSTAT_KIND_LOCK); else if (strcmp(target, "recovery_prefetch") == 0) XLogPrefetchResetStats(); else if (strcmp(target, "slru") == 0) diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 83f6501df38..c224e842e32 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6026,6 +6026,15 @@ proargnames => '{backend_type,object,context,reads,read_bytes,read_time,writes,write_bytes,write_time,writebacks,writeback_time,extends,extend_bytes,extend_time,hits,evictions,reuses,fsyncs,fsync_time,stats_reset}', prosrc => 'pg_stat_get_io' }, +{ oid => '9375', descr => 'statistics: per lock type statistics', + proname => 'pg_stat_get_lock', prorows => '10', proretset => 't', + provolatile => 'v', proparallel => 'r', prorettype => 'record', + proargtypes => '', + proallargtypes => '{text,int8,int8,int8,timestamptz}', + proargmodes => '{o,o,o,o,o}', + proargnames => '{locktype,waits,wait_time,fastpath_exceeded,stats_reset}', + prosrc => 'pg_stat_get_lock' }, + { oid => '6386', descr => 'statistics: backend IO statistics', proname => 'pg_stat_get_backend_io', prorows => '5', proretset => 't', provolatile => 'v', proparallel => 'r', prorettype => 'record', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index dfbead474b1..175e431ed81 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -641,6 +641,7 @@ extern void pgstat_lock_flush(bool nowait); extern void pgstat_count_lock_waits(uint8 locktag_type); extern void pgstat_count_lock_fastpath_exceeded(uint8 locktag_type); extern void pgstat_count_lock_wait_time(uint8 locktag_type, long msecs); +extern PgStat_Lock *pgstat_fetch_stat_lock(void); /* * Functions in pgstat_database.c diff --git a/src/test/isolation/expected/stats.out b/src/test/isolation/expected/stats.out index cfad309ccf3..d3cb8715d95 100644 --- a/src/test/isolation/expected/stats.out +++ b/src/test/isolation/expected/stats.out @@ -3751,3 +3751,150 @@ test_stat_func| 1|t |t (1 row) step s1_commit: COMMIT; + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_begin s1_lock_relation s2_begin s2_ff s2_lock_relation s1_sleep s1_commit s2_commit s2_report_stat_lock_relation +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_begin: BEGIN; +step s1_lock_relation: LOCK TABLE test_stat_tab; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_relation: LOCK TABLE test_stat_tab; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_lock_relation: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_relation: SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column? +--------+-------- +t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_table_insert s1_begin s1_table_update_k1 s2_begin s2_ff s2_table_update_k1 s1_sleep s1_commit s2_commit s2_report_stat_lock_transactionid +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_table_insert: INSERT INTO test_stat_tab(key, value) VALUES('k1', 1), ('k2', 1), ('k3', 1); +step s1_begin: BEGIN; +step s1_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_table_update_k1: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_transactionid: SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'transactionid'; +?column?|?column? +--------+-------- +t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_lock_advisory_lock s2_begin s2_ff s2_lock_advisory_lock s1_sleep s1_lock_advisory_unlock s2_lock_advisory_unlock s2_commit s2_report_stat_lock_advisory +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_lock_advisory_lock: SELECT pg_advisory_lock(1); +pg_advisory_lock +---------------- + +(1 row) + +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_advisory_lock: SELECT pg_advisory_lock(1); <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_lock_advisory_lock: <... completed> +pg_advisory_lock +---------------- + +(1 row) + +step s2_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_commit: COMMIT; +step s2_report_stat_lock_advisory: SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'advisory'; +?column?|?column? +--------+-------- +t |t +(1 row) + diff --git a/src/test/isolation/expected/stats_1.out b/src/test/isolation/expected/stats_1.out index e1d937784cb..89d2e4cc6b4 100644 --- a/src/test/isolation/expected/stats_1.out +++ b/src/test/isolation/expected/stats_1.out @@ -3775,3 +3775,150 @@ test_stat_func| 1|t |t (1 row) step s1_commit: COMMIT; + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_begin s1_lock_relation s2_begin s2_ff s2_lock_relation s1_sleep s1_commit s2_commit s2_report_stat_lock_relation +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_begin: BEGIN; +step s1_lock_relation: LOCK TABLE test_stat_tab; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_relation: LOCK TABLE test_stat_tab; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_lock_relation: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_relation: SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column? +--------+-------- +t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_table_insert s1_begin s1_table_update_k1 s2_begin s2_ff s2_table_update_k1 s1_sleep s1_commit s2_commit s2_report_stat_lock_transactionid +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_table_insert: INSERT INTO test_stat_tab(key, value) VALUES('k1', 1), ('k2', 1), ('k3', 1); +step s1_begin: BEGIN; +step s1_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_table_update_k1: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_transactionid: SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'transactionid'; +?column?|?column? +--------+-------- +t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_lock_advisory_lock s2_begin s2_ff s2_lock_advisory_lock s1_sleep s1_lock_advisory_unlock s2_lock_advisory_unlock s2_commit s2_report_stat_lock_advisory +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_lock_advisory_lock: SELECT pg_advisory_lock(1); +pg_advisory_lock +---------------- + +(1 row) + +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_advisory_lock: SELECT pg_advisory_lock(1); <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_lock_advisory_lock: <... completed> +pg_advisory_lock +---------------- + +(1 row) + +step s2_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_commit: COMMIT; +step s2_report_stat_lock_advisory: SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'advisory'; +?column?|?column? +--------+-------- +t |t +(1 row) + diff --git a/src/test/isolation/specs/stats.spec b/src/test/isolation/specs/stats.spec index da16710da0f..c600fc11157 100644 --- a/src/test/isolation/specs/stats.spec +++ b/src/test/isolation/specs/stats.spec @@ -130,6 +130,14 @@ step s1_slru_check_stats { WHERE before.stat = 'blks_zeroed'; } +# Lock stats steps +step s1_set_deadlock_timeout { SET deadlock_timeout = '10ms'; } +step s1_set_log_lock_waits { SET log_lock_waits = on; } +step s1_reset_stat_lock { SELECT pg_stat_reset_shared('lock'); } +step s1_sleep { SELECT pg_sleep(0.5); } +step s1_lock_relation { LOCK TABLE test_stat_tab; } +step s1_lock_advisory_lock { SELECT pg_advisory_lock(1); } +step s1_lock_advisory_unlock { SELECT pg_advisory_unlock(1); } session s2 setup { SET stats_fetch_consistency = 'none'; } @@ -164,6 +172,15 @@ step s2_big_notify { SELECT pg_notify('stats_test_use', repeat(i::text, current_setting('block_size')::int / 2)) FROM generate_series(1, 3) g(i); } +# Lock stats steps +step s2_set_deadlock_timeout { SET deadlock_timeout = '10ms'; } +step s2_set_log_lock_waits { SET log_lock_waits = on; } +step s2_report_stat_lock_relation { SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; } +step s2_report_stat_lock_transactionid { SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'transactionid'; } +step s2_report_stat_lock_advisory { SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'advisory'; } +step s2_lock_relation { LOCK TABLE test_stat_tab; } +step s2_lock_advisory_lock { SELECT pg_advisory_lock(1); } +step s2_lock_advisory_unlock { SELECT pg_advisory_unlock(1); } ###################### # Function stats tests @@ -765,3 +782,62 @@ permutation s1_clear_snapshot s1_func_stats s1_commit + +###################### +# Lock stats tests +###################### + +# relation lock + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_set_log_lock_waits + s1_begin + s1_lock_relation + s2_begin + s2_ff + s2_lock_relation + s1_sleep + s1_commit + s2_commit + s2_report_stat_lock_relation + +# transaction lock + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_set_log_lock_waits + s1_table_insert + s1_begin + s1_table_update_k1 + s2_begin + s2_ff + s2_table_update_k1 + s1_sleep + s1_commit + s2_commit + s2_report_stat_lock_transactionid + +# advisory lock + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_set_log_lock_waits + s1_lock_advisory_lock + s2_begin + s2_ff + s2_lock_advisory_lock + s1_sleep + s1_lock_advisory_unlock + s2_lock_advisory_unlock + s2_commit + s2_report_stat_lock_advisory diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index 78a37d9fc8f..c013fbebb88 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1951,6 +1951,12 @@ pg_stat_io| SELECT backend_type, fsync_time, stats_reset FROM pg_stat_get_io() b(backend_type, object, context, reads, read_bytes, read_time, writes, write_bytes, write_time, writebacks, writeback_time, extends, extend_bytes, extend_time, hits, evictions, reuses, fsyncs, fsync_time, stats_reset); +pg_stat_lock| SELECT locktype, + waits, + wait_time, + fastpath_exceeded, + stats_reset + FROM pg_stat_get_lock() l(locktype, waits, wait_time, fastpath_exceeded, stats_reset); pg_stat_progress_analyze| SELECT s.pid, s.datid, d.datname, diff --git a/src/test/regress/expected/stats.out b/src/test/regress/expected/stats.out index cd00f35bf7a..ddb016e73d2 100644 --- a/src/test/regress/expected/stats.out +++ b/src/test/regress/expected/stats.out @@ -1910,4 +1910,46 @@ SELECT * FROM check_estimated_rows('SELECT * FROM table_fillfactor'); (1 row) DROP TABLE table_fillfactor; +-- Test fastpath_exceeded stat +CREATE TABLE part_test (id int) PARTITION BY RANGE (id); +SELECT pg_stat_reset_shared('lock'); + pg_stat_reset_shared +---------------------- + +(1 row) + +-- Create 70 partitions (exceeds default number of slots) +DO $$ +BEGIN + FOR i IN 1..70 LOOP + EXECUTE format( + 'CREATE TABLE part_test_%s PARTITION OF part_test + FOR VALUES FROM (%s) TO (%s)', + i, (i-1)*1000, i*1000 + ); + END LOOP; +END; +$$; +SELECT fastpath_exceeded AS fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation' \gset +-- Needs a lock on each partition +SELECT count(*) FROM part_test; + count +------- + 0 +(1 row) + +-- Ensure pending stats are flushed +SELECT pg_stat_force_next_flush(); + pg_stat_force_next_flush +-------------------------- + +(1 row) + +SELECT fastpath_exceeded > :fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation'; + ?column? +---------- + t +(1 row) + +DROP TABLE part_test; -- End of Stats Test diff --git a/src/test/regress/sql/stats.sql b/src/test/regress/sql/stats.sql index 8768e0f27fd..fac7d734758 100644 --- a/src/test/regress/sql/stats.sql +++ b/src/test/regress/sql/stats.sql @@ -944,4 +944,34 @@ SELECT * FROM check_estimated_rows('SELECT * FROM table_fillfactor'); DROP TABLE table_fillfactor; +-- Test fastpath_exceeded stat +CREATE TABLE part_test (id int) PARTITION BY RANGE (id); + +SELECT pg_stat_reset_shared('lock'); + +-- Create 70 partitions (exceeds default number of slots) +DO $$ +BEGIN + FOR i IN 1..70 LOOP + EXECUTE format( + 'CREATE TABLE part_test_%s PARTITION OF part_test + FOR VALUES FROM (%s) TO (%s)', + i, (i-1)*1000, i*1000 + ); + END LOOP; +END; +$$; + +SELECT fastpath_exceeded AS fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation' \gset + +-- Needs a lock on each partition +SELECT count(*) FROM part_test; + +-- Ensure pending stats are flushed +SELECT pg_stat_force_next_flush(); + +SELECT fastpath_exceeded > :fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation'; + +DROP TABLE part_test; + -- End of Stats Test -- 2.34.1 --n9KlQA279nLbrjqm-- ^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v7 2/3] Add the pg_stat_lock view @ 2025-07-31 09:35 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 14+ messages in thread From: Bertrand Drouvot @ 2025-07-31 09:35 UTC (permalink / raw) This new view reports lock statistics. Note that it does not omit combinations which do not make sense (as pg_locks does). Also wait_time is reported as bigint as the deadlock_timeout default value is 1s. This commit also adds documentation and a few tests. XXX: Bump catversion --- doc/src/sgml/monitoring.sgml | 125 +++++++++++++++ src/backend/catalog/system_views.sql | 10 ++ src/backend/utils/activity/pgstat_lock.c | 8 + src/backend/utils/adt/pgstatfuncs.c | 40 +++++ src/include/catalog/pg_proc.dat | 9 ++ src/include/pgstat.h | 1 + src/test/isolation/expected/stats.out | 189 +++++++++++++++++++++++ src/test/isolation/expected/stats_1.out | 189 +++++++++++++++++++++++ src/test/isolation/specs/stats.spec | 95 ++++++++++++ src/test/regress/expected/rules.out | 7 + src/test/regress/expected/stats.out | 48 ++++++ src/test/regress/sql/stats.sql | 36 +++++ 12 files changed, 757 insertions(+) 16.7% doc/src/sgml/ 5.0% src/backend/utils/adt/ 53.6% src/test/isolation/expected/ 11.1% src/test/isolation/specs/ 5.6% src/test/regress/expected/ 3.9% src/test/regress/sql/ 3.7% src/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index b77d189a500..cfcb8189418 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -493,6 +493,15 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser </entry> </row> + <row> + <entry><structname>pg_stat_lock</structname><indexterm><primary>pg_stat_lock</primary></indexterm></entry> + <entry> + One row for each lock type, containing cluster-wide locks statistics. + See <link linkend="monitoring-pg-stat-lock-view"> + <structname>pg_stat_lock</structname></link> for details. + </entry> + </row> + <row> <entry><structname>pg_stat_replication_slots</structname><indexterm><primary>pg_stat_replication_slots</primary></indexterm></entry> <entry>One row per replication slot, showing statistics about the @@ -3124,6 +3133,116 @@ description | Waiting for a newly initialized WAL file to reach durable storage </sect2> + + <sect2 id="monitoring-pg-stat-lock-view"> + <title><structname>pg_stat_lock</structname></title> + + <indexterm> + <primary>pg_stat_lock</primary> + </indexterm> + + <para> + The <structname>pg_stat_lock</structname> view will contain one row for each + lock type, showing cluster-wide locks statistics. + </para> + + <table id="pg-stat-lock-view" xreflabel="pg_stat_lock"> + <title><structname>pg_stat_lock</structname> View</title> + <tgroup cols="1"> + <thead> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + Column Type + </para> + <para> + Description + </para> + </entry> + </row> + </thead> + <tbody> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>locktype</structfield> <type>text</type> + </para> + <para> + Type of the lockable object. See <link linkend="view-pg-locks"> + <structname>pg_locks</structname></link> for details. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>waits</structfield> <type>bigint</type> + </para> + <para> + Number of times a lock of this type had to wait because of a + conflicting lock. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>timed_waits</structfield> <type>bigint</type> + </para> + <para> + Number of times a lock of this type had to wait because of a + conflicting lock. Only incremented when <xref linkend="guc-log-lock-waits"/> + is enabled and the lock was successfully acquired after waiting longer + than <xref linkend="guc-deadlock-timeout"/>. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>wait_time</structfield> <type>bigint</type> + </para> + <para> + Total time spent waiting for locks of this type, in milliseconds. + Only incremented when <xref linkend="guc-log-lock-waits"/> is enabled and + the lock was successfully acquired after waiting longer than + <xref linkend="guc-deadlock-timeout"/>. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>fastpath_exceeded</structfield> <type>bigint</type> + </para> + <para> + Number of times a lock of this type could not be acquired via fast path + because the fast path slot limit was exceeded. You may want to increase + <xref linkend="guc-max-locks-per-transaction"/> if you feel this counter + is too high. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>stats_reset</structfield> <type>timestamp with time zone</type> + </para> + <para> + Time at which these statistics were last reset. + </para> + </entry> + </row> + </tbody> + </tgroup> + </table> + </sect2> + <sect2 id="monitoring-pg-stat-bgwriter-view"> <title><structname>pg_stat_bgwriter</structname></title> @@ -5195,6 +5314,12 @@ description | Waiting for a newly initialized WAL file to reach durable storage <structname>pg_stat_io</structname> view. </para> </listitem> + <listitem> + <para> + <literal>lock</literal>: Reset all the counters shown in the + <structname>pg_stat_lock</structname> view. + </para> + </listitem> <listitem> <para> <literal>recovery_prefetch</literal>: Reset all the counters shown in diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 1ea8f1faa9e..dcf12a202e0 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -985,6 +985,16 @@ CREATE VIEW pg_stat_slru AS s.stats_reset FROM pg_stat_get_slru() s; +CREATE VIEW pg_stat_lock AS + SELECT + l.locktype, + l.waits, + l.timed_waits, + l.wait_time, + l.fastpath_exceeded, + l.stats_reset + FROM pg_stat_get_lock() l; + CREATE VIEW pg_stat_wal_receiver AS SELECT s.pid, diff --git a/src/backend/utils/activity/pgstat_lock.c b/src/backend/utils/activity/pgstat_lock.c index b410f376d49..451fba4f8d9 100644 --- a/src/backend/utils/activity/pgstat_lock.c +++ b/src/backend/utils/activity/pgstat_lock.c @@ -21,6 +21,14 @@ static PgStat_PendingLock PendingLockStats; static bool have_lockstats = false; +PgStat_Lock * +pgstat_fetch_stat_lock(void) +{ + pgstat_snapshot_fixed(PGSTAT_KIND_LOCK); + + return &pgStatLocal.snapshot.lock; +} + /* * Simpler wrapper of pgstat_lock_flush_cb() */ diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index b1df96e7b0b..3b7cbdb82be 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1737,6 +1737,43 @@ pg_stat_get_wal(PG_FUNCTION_ARGS) wal_stats->stat_reset_timestamp)); } +Datum +pg_stat_get_lock(PG_FUNCTION_ARGS) +{ +#define PG_STAT_LOCK_COLS 6 + ReturnSetInfo *rsinfo; + PgStat_Lock *lock_stats; + + InitMaterializedSRF(fcinfo, 0); + rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; + + lock_stats = pgstat_fetch_stat_lock(); + + for (int lcktype = 0; lcktype <= LOCKTAG_LAST_TYPE; lcktype++) + { + const char *locktypename; + Datum values[PG_STAT_LOCK_COLS] = {0}; + bool nulls[PG_STAT_LOCK_COLS] = {0}; + PgStat_LockEntry *lck_stats = &lock_stats->stats[lcktype]; + int i = 0; + + locktypename = LockTagTypeNames[lcktype]; + + values[i++] = CStringGetTextDatum(locktypename); + values[i++] = Int64GetDatum(lck_stats->waits); + values[i++] = Int64GetDatum(lck_stats->timed_waits); + values[i++] = Int64GetDatum(lck_stats->wait_time); + values[i++] = Int64GetDatum(lck_stats->fastpath_exceeded); + values[i] = TimestampTzGetDatum(lock_stats->stat_reset_timestamp); + + Assert(i + 1 == PG_STAT_LOCK_COLS); + + tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls); + } + + return (Datum) 0; +} + /* * Returns statistics of SLRU caches. */ @@ -1921,6 +1958,7 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) pgstat_reset_of_kind(PGSTAT_KIND_BGWRITER); pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER); pgstat_reset_of_kind(PGSTAT_KIND_IO); + pgstat_reset_of_kind(PGSTAT_KIND_LOCK); XLogPrefetchResetStats(); pgstat_reset_of_kind(PGSTAT_KIND_SLRU); pgstat_reset_of_kind(PGSTAT_KIND_WAL); @@ -1938,6 +1976,8 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER); else if (strcmp(target, "io") == 0) pgstat_reset_of_kind(PGSTAT_KIND_IO); + else if (strcmp(target, "lock") == 0) + pgstat_reset_of_kind(PGSTAT_KIND_LOCK); else if (strcmp(target, "recovery_prefetch") == 0) XLogPrefetchResetStats(); else if (strcmp(target, "slru") == 0) diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index dac40992cbc..3cd497a8e1f 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6027,6 +6027,15 @@ proargnames => '{backend_type,object,context,reads,read_bytes,read_time,writes,write_bytes,write_time,writebacks,writeback_time,extends,extend_bytes,extend_time,hits,evictions,reuses,fsyncs,fsync_time,stats_reset}', prosrc => 'pg_stat_get_io' }, +{ oid => '9375', descr => 'statistics: per lock type statistics', + proname => 'pg_stat_get_lock', prorows => '10', proretset => 't', + provolatile => 'v', proparallel => 'r', prorettype => 'record', + proargtypes => '', + proallargtypes => '{text,int8,int8,int8,int8,timestamptz}', + proargmodes => '{o,o,o,o,o,o}', + proargnames => '{locktype,waits,timed_waits,wait_time,fastpath_exceeded,stats_reset}', + prosrc => 'pg_stat_get_lock' }, + { oid => '6386', descr => 'statistics: backend IO statistics', proname => 'pg_stat_get_backend_io', prorows => '5', proretset => 't', provolatile => 'v', proparallel => 'r', prorettype => 'record', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 9c4508d5544..a4cc006794d 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -641,6 +641,7 @@ extern void pgstat_lock_flush(bool nowait); extern void pgstat_count_lock_waits(uint8 locktag_type); extern void pgstat_count_lock_fastpath_exceeded(uint8 locktag_type); extern void pgstat_count_lock_timed_wait(uint8 locktag_type, long msecs); +extern PgStat_Lock *pgstat_fetch_stat_lock(void); /* * Functions in pgstat_database.c diff --git a/src/test/isolation/expected/stats.out b/src/test/isolation/expected/stats.out index cfad309ccf3..86d70a0705c 100644 --- a/src/test/isolation/expected/stats.out +++ b/src/test/isolation/expected/stats.out @@ -3751,3 +3751,192 @@ test_stat_func| 1|t |t (1 row) step s1_commit: COMMIT; + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_begin s1_lock_relation s2_begin s2_ff s2_lock_relation s1_sleep s1_commit s2_commit s2_report_stat_lock_relation +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_begin: BEGIN; +step s1_lock_relation: LOCK TABLE test_stat_tab; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_relation: LOCK TABLE test_stat_tab; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_lock_relation: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_relation: SELECT waits > 0, timed_waits = waits, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column?|?column? +--------+--------+-------- +t |t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_table_insert s1_begin s1_table_update_k1 s2_begin s2_ff s2_table_update_k1 s1_sleep s1_commit s2_commit s2_report_stat_lock_transactionid +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_table_insert: INSERT INTO test_stat_tab(key, value) VALUES('k1', 1), ('k2', 1), ('k3', 1); +step s1_begin: BEGIN; +step s1_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_table_update_k1: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_transactionid: SELECT waits > 0, timed_waits = waits, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'transactionid'; +?column?|?column?|?column? +--------+--------+-------- +t |t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_lock_advisory_lock s2_begin s2_ff s2_lock_advisory_lock s1_sleep s1_lock_advisory_unlock s2_lock_advisory_unlock s2_commit s2_report_stat_lock_advisory +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_lock_advisory_lock: SELECT pg_advisory_lock(1); +pg_advisory_lock +---------------- + +(1 row) + +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_advisory_lock: SELECT pg_advisory_lock(1); <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_lock_advisory_lock: <... completed> +pg_advisory_lock +---------------- + +(1 row) + +step s2_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_commit: COMMIT; +step s2_report_stat_lock_advisory: SELECT waits > 0, timed_waits = waits, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'advisory'; +?column?|?column?|?column? +--------+--------+-------- +t |t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_unset_log_lock_waits s1_begin s1_lock_relation s2_begin s2_ff s2_lock_relation s1_sleep s1_commit s2_commit s2_report_stat_lock_relation +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_unset_log_lock_waits: SET log_lock_waits = off; +step s1_begin: BEGIN; +step s1_lock_relation: LOCK TABLE test_stat_tab; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_relation: LOCK TABLE test_stat_tab; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_lock_relation: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_relation: SELECT waits > 0, timed_waits = waits, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column?|?column? +--------+--------+-------- +t |f |f +(1 row) + diff --git a/src/test/isolation/expected/stats_1.out b/src/test/isolation/expected/stats_1.out index e1d937784cb..4e4c68b4525 100644 --- a/src/test/isolation/expected/stats_1.out +++ b/src/test/isolation/expected/stats_1.out @@ -3775,3 +3775,192 @@ test_stat_func| 1|t |t (1 row) step s1_commit: COMMIT; + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_begin s1_lock_relation s2_begin s2_ff s2_lock_relation s1_sleep s1_commit s2_commit s2_report_stat_lock_relation +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_begin: BEGIN; +step s1_lock_relation: LOCK TABLE test_stat_tab; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_relation: LOCK TABLE test_stat_tab; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_lock_relation: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_relation: SELECT waits > 0, timed_waits = waits, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column?|?column? +--------+--------+-------- +t |t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_table_insert s1_begin s1_table_update_k1 s2_begin s2_ff s2_table_update_k1 s1_sleep s1_commit s2_commit s2_report_stat_lock_transactionid +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_table_insert: INSERT INTO test_stat_tab(key, value) VALUES('k1', 1), ('k2', 1), ('k3', 1); +step s1_begin: BEGIN; +step s1_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_table_update_k1: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_transactionid: SELECT waits > 0, timed_waits = waits, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'transactionid'; +?column?|?column?|?column? +--------+--------+-------- +t |t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_lock_advisory_lock s2_begin s2_ff s2_lock_advisory_lock s1_sleep s1_lock_advisory_unlock s2_lock_advisory_unlock s2_commit s2_report_stat_lock_advisory +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_lock_advisory_lock: SELECT pg_advisory_lock(1); +pg_advisory_lock +---------------- + +(1 row) + +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_advisory_lock: SELECT pg_advisory_lock(1); <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_lock_advisory_lock: <... completed> +pg_advisory_lock +---------------- + +(1 row) + +step s2_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_commit: COMMIT; +step s2_report_stat_lock_advisory: SELECT waits > 0, timed_waits = waits, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'advisory'; +?column?|?column?|?column? +--------+--------+-------- +t |t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_unset_log_lock_waits s1_begin s1_lock_relation s2_begin s2_ff s2_lock_relation s1_sleep s1_commit s2_commit s2_report_stat_lock_relation +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_unset_log_lock_waits: SET log_lock_waits = off; +step s1_begin: BEGIN; +step s1_lock_relation: LOCK TABLE test_stat_tab; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_relation: LOCK TABLE test_stat_tab; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_lock_relation: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_relation: SELECT waits > 0, timed_waits = waits, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column?|?column? +--------+--------+-------- +t |f |f +(1 row) + diff --git a/src/test/isolation/specs/stats.spec b/src/test/isolation/specs/stats.spec index da16710da0f..d4b53545198 100644 --- a/src/test/isolation/specs/stats.spec +++ b/src/test/isolation/specs/stats.spec @@ -130,6 +130,14 @@ step s1_slru_check_stats { WHERE before.stat = 'blks_zeroed'; } +# Lock stats steps +step s1_set_deadlock_timeout { SET deadlock_timeout = '10ms'; } +step s1_set_log_lock_waits { SET log_lock_waits = on; } +step s1_reset_stat_lock { SELECT pg_stat_reset_shared('lock'); } +step s1_sleep { SELECT pg_sleep(0.5); } +step s1_lock_relation { LOCK TABLE test_stat_tab; } +step s1_lock_advisory_lock { SELECT pg_advisory_lock(1); } +step s1_lock_advisory_unlock { SELECT pg_advisory_unlock(1); } session s2 setup { SET stats_fetch_consistency = 'none'; } @@ -164,6 +172,16 @@ step s2_big_notify { SELECT pg_notify('stats_test_use', repeat(i::text, current_setting('block_size')::int / 2)) FROM generate_series(1, 3) g(i); } +# Lock stats steps +step s2_set_deadlock_timeout { SET deadlock_timeout = '10ms'; } +step s2_set_log_lock_waits { SET log_lock_waits = on; } +step s2_unset_log_lock_waits { SET log_lock_waits = off; } +step s2_report_stat_lock_relation { SELECT waits > 0, timed_waits = waits, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; } +step s2_report_stat_lock_transactionid { SELECT waits > 0, timed_waits = waits, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'transactionid'; } +step s2_report_stat_lock_advisory { SELECT waits > 0, timed_waits = waits, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'advisory'; } +step s2_lock_relation { LOCK TABLE test_stat_tab; } +step s2_lock_advisory_lock { SELECT pg_advisory_lock(1); } +step s2_lock_advisory_unlock { SELECT pg_advisory_unlock(1); } ###################### # Function stats tests @@ -765,3 +783,80 @@ permutation s1_clear_snapshot s1_func_stats s1_commit + +###################### +# Lock stats tests +###################### + +# relation lock + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_set_log_lock_waits + s1_begin + s1_lock_relation + s2_begin + s2_ff + s2_lock_relation + s1_sleep + s1_commit + s2_commit + s2_report_stat_lock_relation + +# transaction lock + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_set_log_lock_waits + s1_table_insert + s1_begin + s1_table_update_k1 + s2_begin + s2_ff + s2_table_update_k1 + s1_sleep + s1_commit + s2_commit + s2_report_stat_lock_transactionid + +# advisory lock + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_set_log_lock_waits + s1_lock_advisory_lock + s2_begin + s2_ff + s2_lock_advisory_lock + s1_sleep + s1_lock_advisory_unlock + s2_lock_advisory_unlock + s2_commit + s2_report_stat_lock_advisory + +# Ensure log_lock_waits behaves correctly + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_unset_log_lock_waits + s1_begin + s1_lock_relation + s2_begin + s2_ff + s2_lock_relation + s1_sleep + s1_commit + s2_commit + s2_report_stat_lock_relation diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index 78a37d9fc8f..2fcb48df6c9 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1951,6 +1951,13 @@ pg_stat_io| SELECT backend_type, fsync_time, stats_reset FROM pg_stat_get_io() b(backend_type, object, context, reads, read_bytes, read_time, writes, write_bytes, write_time, writebacks, writeback_time, extends, extend_bytes, extend_time, hits, evictions, reuses, fsyncs, fsync_time, stats_reset); +pg_stat_lock| SELECT locktype, + waits, + timed_waits, + wait_time, + fastpath_exceeded, + stats_reset + FROM pg_stat_get_lock() l(locktype, waits, timed_waits, wait_time, fastpath_exceeded, stats_reset); pg_stat_progress_analyze| SELECT s.pid, s.datid, d.datname, diff --git a/src/test/regress/expected/stats.out b/src/test/regress/expected/stats.out index cd00f35bf7a..93de87747d4 100644 --- a/src/test/regress/expected/stats.out +++ b/src/test/regress/expected/stats.out @@ -1910,4 +1910,52 @@ SELECT * FROM check_estimated_rows('SELECT * FROM table_fillfactor'); (1 row) DROP TABLE table_fillfactor; +-- Test fastpath_exceeded stat +CREATE TABLE part_test (id int) PARTITION BY RANGE (id); +SELECT pg_stat_reset_shared('lock'); + pg_stat_reset_shared +---------------------- + +(1 row) + +-- Create partitions (exceeds number of slots) +DO $$ +DECLARE + max_locks int; +BEGIN + SELECT setting::int INTO max_locks + FROM pg_settings + WHERE name = 'max_locks_per_transaction'; + + FOR i IN 1..(max_locks + 10) LOOP + EXECUTE format( + 'CREATE TABLE part_test_%s PARTITION OF part_test + FOR VALUES FROM (%s) TO (%s)', + i, (i-1)*1000, i*1000 + ); + END LOOP; +END; +$$; +SELECT fastpath_exceeded AS fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation' \gset +-- Needs a lock on each partition +SELECT count(*) FROM part_test; + count +------- + 0 +(1 row) + +-- Ensure pending stats are flushed +SELECT pg_stat_force_next_flush(); + pg_stat_force_next_flush +-------------------------- + +(1 row) + +SELECT fastpath_exceeded > :fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation'; + ?column? +---------- + t +(1 row) + +DROP TABLE part_test; -- End of Stats Test diff --git a/src/test/regress/sql/stats.sql b/src/test/regress/sql/stats.sql index 8768e0f27fd..db9948eedb8 100644 --- a/src/test/regress/sql/stats.sql +++ b/src/test/regress/sql/stats.sql @@ -944,4 +944,40 @@ SELECT * FROM check_estimated_rows('SELECT * FROM table_fillfactor'); DROP TABLE table_fillfactor; +-- Test fastpath_exceeded stat +CREATE TABLE part_test (id int) PARTITION BY RANGE (id); + +SELECT pg_stat_reset_shared('lock'); + +-- Create partitions (exceeds number of slots) +DO $$ +DECLARE + max_locks int; +BEGIN + SELECT setting::int INTO max_locks + FROM pg_settings + WHERE name = 'max_locks_per_transaction'; + + FOR i IN 1..(max_locks + 10) LOOP + EXECUTE format( + 'CREATE TABLE part_test_%s PARTITION OF part_test + FOR VALUES FROM (%s) TO (%s)', + i, (i-1)*1000, i*1000 + ); + END LOOP; +END; +$$; + +SELECT fastpath_exceeded AS fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation' \gset + +-- Needs a lock on each partition +SELECT count(*) FROM part_test; + +-- Ensure pending stats are flushed +SELECT pg_stat_force_next_flush(); + +SELECT fastpath_exceeded > :fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation'; + +DROP TABLE part_test; + -- End of Stats Test -- 2.34.1 --gJ14Jcshhoq1cFBD Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v7-0003-Introduce-a-new-track_lock_timing-GUC.patch" ^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v9 2/3] Add the pg_stat_lock view @ 2025-07-31 09:35 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 14+ messages in thread From: Bertrand Drouvot @ 2025-07-31 09:35 UTC (permalink / raw) This new view reports lock statistics. Note that it does not omit combinations which do not make sense (as pg_locks does). Also wait_time is reported as bigint as the deadlock_timeout default value is 1s. This commit also adds documentation and a few tests. XXX: Bump catversion --- doc/src/sgml/monitoring.sgml | 113 ++++++++++++++ src/backend/catalog/system_views.sql | 9 ++ src/backend/utils/activity/pgstat_lock.c | 8 + src/backend/utils/adt/pgstatfuncs.c | 39 +++++ src/include/catalog/pg_proc.dat | 9 ++ src/include/pgstat.h | 1 + src/test/isolation/expected/stats.out | 189 +++++++++++++++++++++++ src/test/isolation/expected/stats_1.out | 189 +++++++++++++++++++++++ src/test/isolation/specs/stats.spec | 95 ++++++++++++ src/test/regress/expected/rules.out | 6 + src/test/regress/expected/stats.out | 48 ++++++ src/test/regress/sql/stats.sql | 36 +++++ 12 files changed, 742 insertions(+) 15.9% doc/src/sgml/ 5.0% src/backend/utils/adt/ 54.1% src/test/isolation/expected/ 11.3% src/test/isolation/specs/ 5.7% src/test/regress/expected/ 4.0% src/test/regress/sql/ 3.6% src/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 462019a972c..221ee28cc7b 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -509,6 +509,15 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser </entry> </row> + <row> + <entry><structname>pg_stat_lock</structname><indexterm><primary>pg_stat_lock</primary></indexterm></entry> + <entry> + One row for each lock type, containing cluster-wide locks statistics. + See <link linkend="monitoring-pg-stat-lock-view"> + <structname>pg_stat_lock</structname></link> for details. + </entry> + </row> + <row> <entry><structname>pg_stat_replication_slots</structname><indexterm><primary>pg_stat_replication_slots</primary></indexterm></entry> <entry>One row per replication slot, showing statistics about the @@ -3283,6 +3292,104 @@ description | Waiting for a newly initialized WAL file to reach durable storage </sect2> + + <sect2 id="monitoring-pg-stat-lock-view"> + <title><structname>pg_stat_lock</structname></title> + + <indexterm> + <primary>pg_stat_lock</primary> + </indexterm> + + <para> + The <structname>pg_stat_lock</structname> view will contain one row for each + lock type, showing cluster-wide locks statistics. + </para> + + <table id="pg-stat-lock-view" xreflabel="pg_stat_lock"> + <title><structname>pg_stat_lock</structname> View</title> + <tgroup cols="1"> + <thead> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + Column Type + </para> + <para> + Description + </para> + </entry> + </row> + </thead> + <tbody> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>locktype</structfield> <type>text</type> + </para> + <para> + Type of the lockable object. See <link linkend="view-pg-locks"> + <structname>pg_locks</structname></link> for details. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>waits</structfield> <type>bigint</type> + </para> + <para> + Number of times a lock of this type had to wait because of a + conflicting lock. Only incremented when <xref linkend="guc-log-lock-waits"/> + is enabled and the lock was successfully acquired after waiting longer + than <xref linkend="guc-deadlock-timeout"/>. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>wait_time</structfield> <type>bigint</type> + </para> + <para> + Total time spent waiting for locks of this type, in milliseconds. + Only incremented when <xref linkend="guc-log-lock-waits"/> is enabled and + the lock was successfully acquired after waiting longer than + <xref linkend="guc-deadlock-timeout"/>. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>fastpath_exceeded</structfield> <type>bigint</type> + </para> + <para> + Number of times a lock of this type could not be acquired via fast path + because the fast path slot limit was exceeded. You may want to increase + <xref linkend="guc-max-locks-per-transaction"/> if you feel this counter + is too high. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>stats_reset</structfield> <type>timestamp with time zone</type> + </para> + <para> + Time at which these statistics were last reset. + </para> + </entry> + </row> + </tbody> + </tgroup> + </table> + </sect2> + <sect2 id="monitoring-pg-stat-bgwriter-view"> <title><structname>pg_stat_bgwriter</structname></title> @@ -5372,6 +5479,12 @@ description | Waiting for a newly initialized WAL file to reach durable storage <structname>pg_stat_io</structname> view. </para> </listitem> + <listitem> + <para> + <literal>lock</literal>: Reset all the counters shown in the + <structname>pg_stat_lock</structname> view. + </para> + </listitem> <listitem> <para> <literal>recovery_prefetch</literal>: Reset all the counters shown in diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index f1ed7b58f13..e54018004db 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -985,6 +985,15 @@ CREATE VIEW pg_stat_slru AS s.stats_reset FROM pg_stat_get_slru() s; +CREATE VIEW pg_stat_lock AS + SELECT + l.locktype, + l.waits, + l.wait_time, + l.fastpath_exceeded, + l.stats_reset + FROM pg_stat_get_lock() l; + CREATE VIEW pg_stat_wal_receiver AS SELECT s.pid, diff --git a/src/backend/utils/activity/pgstat_lock.c b/src/backend/utils/activity/pgstat_lock.c index 91a53a6013a..954fa51c3e0 100644 --- a/src/backend/utils/activity/pgstat_lock.c +++ b/src/backend/utils/activity/pgstat_lock.c @@ -21,6 +21,14 @@ static PgStat_PendingLock PendingLockStats; static bool have_lockstats = false; +PgStat_Lock * +pgstat_fetch_stat_lock(void) +{ + pgstat_snapshot_fixed(PGSTAT_KIND_LOCK); + + return &pgStatLocal.snapshot.lock; +} + /* * Simpler wrapper of pgstat_lock_flush_cb() */ diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 5f907335990..9185a8e6b83 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1741,6 +1741,42 @@ pg_stat_get_wal(PG_FUNCTION_ARGS) wal_stats->stat_reset_timestamp)); } +Datum +pg_stat_get_lock(PG_FUNCTION_ARGS) +{ +#define PG_STAT_LOCK_COLS 5 + ReturnSetInfo *rsinfo; + PgStat_Lock *lock_stats; + + InitMaterializedSRF(fcinfo, 0); + rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; + + lock_stats = pgstat_fetch_stat_lock(); + + for (int lcktype = 0; lcktype <= LOCKTAG_LAST_TYPE; lcktype++) + { + const char *locktypename; + Datum values[PG_STAT_LOCK_COLS] = {0}; + bool nulls[PG_STAT_LOCK_COLS] = {0}; + PgStat_LockEntry *lck_stats = &lock_stats->stats[lcktype]; + int i = 0; + + locktypename = LockTagTypeNames[lcktype]; + + values[i++] = CStringGetTextDatum(locktypename); + values[i++] = Int64GetDatum(lck_stats->waits); + values[i++] = Int64GetDatum(lck_stats->wait_time); + values[i++] = Int64GetDatum(lck_stats->fastpath_exceeded); + values[i] = TimestampTzGetDatum(lock_stats->stat_reset_timestamp); + + Assert(i + 1 == PG_STAT_LOCK_COLS); + + tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls); + } + + return (Datum) 0; +} + /* * Returns statistics of SLRU caches. */ @@ -1925,6 +1961,7 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) pgstat_reset_of_kind(PGSTAT_KIND_BGWRITER); pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER); pgstat_reset_of_kind(PGSTAT_KIND_IO); + pgstat_reset_of_kind(PGSTAT_KIND_LOCK); XLogPrefetchResetStats(); pgstat_reset_of_kind(PGSTAT_KIND_SLRU); pgstat_reset_of_kind(PGSTAT_KIND_WAL); @@ -1942,6 +1979,8 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER); else if (strcmp(target, "io") == 0) pgstat_reset_of_kind(PGSTAT_KIND_IO); + else if (strcmp(target, "lock") == 0) + pgstat_reset_of_kind(PGSTAT_KIND_LOCK); else if (strcmp(target, "recovery_prefetch") == 0) XLogPrefetchResetStats(); else if (strcmp(target, "slru") == 0) diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index fc8d82665b8..3e2c43e9d1b 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6041,6 +6041,15 @@ proargnames => '{backend_type,object,context,reads,read_bytes,read_time,writes,write_bytes,write_time,writebacks,writeback_time,extends,extend_bytes,extend_time,hits,evictions,reuses,fsyncs,fsync_time,stats_reset}', prosrc => 'pg_stat_get_io' }, +{ oid => '9375', descr => 'statistics: per lock type statistics', + proname => 'pg_stat_get_lock', prorows => '10', proretset => 't', + provolatile => 'v', proparallel => 'r', prorettype => 'record', + proargtypes => '', + proallargtypes => '{text,int8,int8,int8,timestamptz}', + proargmodes => '{o,o,o,o,o}', + proargnames => '{locktype,waits,wait_time,fastpath_exceeded,stats_reset}', + prosrc => 'pg_stat_get_lock' }, + { oid => '6386', descr => 'statistics: backend IO statistics', proname => 'pg_stat_get_backend_io', prorows => '5', proretset => 't', provolatile => 'v', proparallel => 'r', prorettype => 'record', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 4a28ea99705..06e2638e788 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -643,6 +643,7 @@ extern bool pgstat_tracks_io_op(BackendType bktype, IOObject io_object, extern void pgstat_lock_flush(bool nowait); extern void pgstat_count_lock_fastpath_exceeded(uint8 locktag_type); extern void pgstat_count_lock_waits(uint8 locktag_type, long msecs); +extern PgStat_Lock *pgstat_fetch_stat_lock(void); /* * Functions in pgstat_database.c diff --git a/src/test/isolation/expected/stats.out b/src/test/isolation/expected/stats.out index cfad309ccf3..3cae3052e40 100644 --- a/src/test/isolation/expected/stats.out +++ b/src/test/isolation/expected/stats.out @@ -3751,3 +3751,192 @@ test_stat_func| 1|t |t (1 row) step s1_commit: COMMIT; + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_begin s1_lock_relation s2_begin s2_ff s2_lock_relation s1_sleep s1_commit s2_commit s2_report_stat_lock_relation +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_begin: BEGIN; +step s1_lock_relation: LOCK TABLE test_stat_tab; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_relation: LOCK TABLE test_stat_tab; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_lock_relation: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_relation: SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column? +--------+-------- +t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_table_insert s1_begin s1_table_update_k1 s2_begin s2_ff s2_table_update_k1 s1_sleep s1_commit s2_commit s2_report_stat_lock_transactionid +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_table_insert: INSERT INTO test_stat_tab(key, value) VALUES('k1', 1), ('k2', 1), ('k3', 1); +step s1_begin: BEGIN; +step s1_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_table_update_k1: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_transactionid: SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'transactionid'; +?column?|?column? +--------+-------- +t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_lock_advisory_lock s2_begin s2_ff s2_lock_advisory_lock s1_sleep s1_lock_advisory_unlock s2_lock_advisory_unlock s2_commit s2_report_stat_lock_advisory +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_lock_advisory_lock: SELECT pg_advisory_lock(1); +pg_advisory_lock +---------------- + +(1 row) + +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_advisory_lock: SELECT pg_advisory_lock(1); <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_lock_advisory_lock: <... completed> +pg_advisory_lock +---------------- + +(1 row) + +step s2_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_commit: COMMIT; +step s2_report_stat_lock_advisory: SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'advisory'; +?column?|?column? +--------+-------- +t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_unset_log_lock_waits s1_begin s1_lock_relation s2_begin s2_ff s2_lock_relation s1_sleep s1_commit s2_commit s2_report_stat_lock_relation +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_unset_log_lock_waits: SET log_lock_waits = off; +step s1_begin: BEGIN; +step s1_lock_relation: LOCK TABLE test_stat_tab; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_relation: LOCK TABLE test_stat_tab; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_lock_relation: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_relation: SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column? +--------+-------- +f |f +(1 row) + diff --git a/src/test/isolation/expected/stats_1.out b/src/test/isolation/expected/stats_1.out index e1d937784cb..ea4fd97a9a5 100644 --- a/src/test/isolation/expected/stats_1.out +++ b/src/test/isolation/expected/stats_1.out @@ -3775,3 +3775,192 @@ test_stat_func| 1|t |t (1 row) step s1_commit: COMMIT; + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_begin s1_lock_relation s2_begin s2_ff s2_lock_relation s1_sleep s1_commit s2_commit s2_report_stat_lock_relation +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_begin: BEGIN; +step s1_lock_relation: LOCK TABLE test_stat_tab; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_relation: LOCK TABLE test_stat_tab; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_lock_relation: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_relation: SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column? +--------+-------- +t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_table_insert s1_begin s1_table_update_k1 s2_begin s2_ff s2_table_update_k1 s1_sleep s1_commit s2_commit s2_report_stat_lock_transactionid +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_table_insert: INSERT INTO test_stat_tab(key, value) VALUES('k1', 1), ('k2', 1), ('k3', 1); +step s1_begin: BEGIN; +step s1_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_table_update_k1: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_transactionid: SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'transactionid'; +?column?|?column? +--------+-------- +t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_lock_advisory_lock s2_begin s2_ff s2_lock_advisory_lock s1_sleep s1_lock_advisory_unlock s2_lock_advisory_unlock s2_commit s2_report_stat_lock_advisory +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_lock_advisory_lock: SELECT pg_advisory_lock(1); +pg_advisory_lock +---------------- + +(1 row) + +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_advisory_lock: SELECT pg_advisory_lock(1); <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_lock_advisory_lock: <... completed> +pg_advisory_lock +---------------- + +(1 row) + +step s2_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_commit: COMMIT; +step s2_report_stat_lock_advisory: SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'advisory'; +?column?|?column? +--------+-------- +t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_unset_log_lock_waits s1_begin s1_lock_relation s2_begin s2_ff s2_lock_relation s1_sleep s1_commit s2_commit s2_report_stat_lock_relation +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_unset_log_lock_waits: SET log_lock_waits = off; +step s1_begin: BEGIN; +step s1_lock_relation: LOCK TABLE test_stat_tab; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_relation: LOCK TABLE test_stat_tab; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_lock_relation: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_relation: SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column? +--------+-------- +f |f +(1 row) + diff --git a/src/test/isolation/specs/stats.spec b/src/test/isolation/specs/stats.spec index da16710da0f..42be68c545f 100644 --- a/src/test/isolation/specs/stats.spec +++ b/src/test/isolation/specs/stats.spec @@ -130,6 +130,14 @@ step s1_slru_check_stats { WHERE before.stat = 'blks_zeroed'; } +# Lock stats steps +step s1_set_deadlock_timeout { SET deadlock_timeout = '10ms'; } +step s1_set_log_lock_waits { SET log_lock_waits = on; } +step s1_reset_stat_lock { SELECT pg_stat_reset_shared('lock'); } +step s1_sleep { SELECT pg_sleep(0.5); } +step s1_lock_relation { LOCK TABLE test_stat_tab; } +step s1_lock_advisory_lock { SELECT pg_advisory_lock(1); } +step s1_lock_advisory_unlock { SELECT pg_advisory_unlock(1); } session s2 setup { SET stats_fetch_consistency = 'none'; } @@ -164,6 +172,16 @@ step s2_big_notify { SELECT pg_notify('stats_test_use', repeat(i::text, current_setting('block_size')::int / 2)) FROM generate_series(1, 3) g(i); } +# Lock stats steps +step s2_set_deadlock_timeout { SET deadlock_timeout = '10ms'; } +step s2_set_log_lock_waits { SET log_lock_waits = on; } +step s2_unset_log_lock_waits { SET log_lock_waits = off; } +step s2_report_stat_lock_relation { SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; } +step s2_report_stat_lock_transactionid { SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'transactionid'; } +step s2_report_stat_lock_advisory { SELECT waits > 0, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'advisory'; } +step s2_lock_relation { LOCK TABLE test_stat_tab; } +step s2_lock_advisory_lock { SELECT pg_advisory_lock(1); } +step s2_lock_advisory_unlock { SELECT pg_advisory_unlock(1); } ###################### # Function stats tests @@ -765,3 +783,80 @@ permutation s1_clear_snapshot s1_func_stats s1_commit + +###################### +# Lock stats tests +###################### + +# relation lock + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_set_log_lock_waits + s1_begin + s1_lock_relation + s2_begin + s2_ff + s2_lock_relation + s1_sleep + s1_commit + s2_commit + s2_report_stat_lock_relation + +# transaction lock + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_set_log_lock_waits + s1_table_insert + s1_begin + s1_table_update_k1 + s2_begin + s2_ff + s2_table_update_k1 + s1_sleep + s1_commit + s2_commit + s2_report_stat_lock_transactionid + +# advisory lock + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_set_log_lock_waits + s1_lock_advisory_lock + s2_begin + s2_ff + s2_lock_advisory_lock + s1_sleep + s1_lock_advisory_unlock + s2_lock_advisory_unlock + s2_commit + s2_report_stat_lock_advisory + +# Ensure log_lock_waits behaves correctly + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_unset_log_lock_waits + s1_begin + s1_lock_relation + s2_begin + s2_ff + s2_lock_relation + s1_sleep + s1_commit + s2_commit + s2_report_stat_lock_relation diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index 32bea58db2c..2b3cf6d8569 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1952,6 +1952,12 @@ pg_stat_io| SELECT backend_type, fsync_time, stats_reset FROM pg_stat_get_io() b(backend_type, object, context, reads, read_bytes, read_time, writes, write_bytes, write_time, writebacks, writeback_time, extends, extend_bytes, extend_time, hits, evictions, reuses, fsyncs, fsync_time, stats_reset); +pg_stat_lock| SELECT locktype, + waits, + wait_time, + fastpath_exceeded, + stats_reset + FROM pg_stat_get_lock() l(locktype, waits, wait_time, fastpath_exceeded, stats_reset); pg_stat_progress_analyze| SELECT s.pid, s.datid, d.datname, diff --git a/src/test/regress/expected/stats.out b/src/test/regress/expected/stats.out index b99462bf946..ea7f7846895 100644 --- a/src/test/regress/expected/stats.out +++ b/src/test/regress/expected/stats.out @@ -1958,4 +1958,52 @@ SELECT * FROM check_estimated_rows('SELECT * FROM table_fillfactor'); (1 row) DROP TABLE table_fillfactor; +-- Test fastpath_exceeded stat +CREATE TABLE part_test (id int) PARTITION BY RANGE (id); +SELECT pg_stat_reset_shared('lock'); + pg_stat_reset_shared +---------------------- + +(1 row) + +-- Create partitions (exceeds number of slots) +DO $$ +DECLARE + max_locks int; +BEGIN + SELECT setting::int INTO max_locks + FROM pg_settings + WHERE name = 'max_locks_per_transaction'; + + FOR i IN 1..(max_locks + 10) LOOP + EXECUTE format( + 'CREATE TABLE part_test_%s PARTITION OF part_test + FOR VALUES FROM (%s) TO (%s)', + i, (i-1)*1000, i*1000 + ); + END LOOP; +END; +$$; +SELECT fastpath_exceeded AS fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation' \gset +-- Needs a lock on each partition +SELECT count(*) FROM part_test; + count +------- + 0 +(1 row) + +-- Ensure pending stats are flushed +SELECT pg_stat_force_next_flush(); + pg_stat_force_next_flush +-------------------------- + +(1 row) + +SELECT fastpath_exceeded > :fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation'; + ?column? +---------- + t +(1 row) + +DROP TABLE part_test; -- End of Stats Test diff --git a/src/test/regress/sql/stats.sql b/src/test/regress/sql/stats.sql index 941222cf0be..65d8968c83e 100644 --- a/src/test/regress/sql/stats.sql +++ b/src/test/regress/sql/stats.sql @@ -964,4 +964,40 @@ SELECT * FROM check_estimated_rows('SELECT * FROM table_fillfactor'); DROP TABLE table_fillfactor; +-- Test fastpath_exceeded stat +CREATE TABLE part_test (id int) PARTITION BY RANGE (id); + +SELECT pg_stat_reset_shared('lock'); + +-- Create partitions (exceeds number of slots) +DO $$ +DECLARE + max_locks int; +BEGIN + SELECT setting::int INTO max_locks + FROM pg_settings + WHERE name = 'max_locks_per_transaction'; + + FOR i IN 1..(max_locks + 10) LOOP + EXECUTE format( + 'CREATE TABLE part_test_%s PARTITION OF part_test + FOR VALUES FROM (%s) TO (%s)', + i, (i-1)*1000, i*1000 + ); + END LOOP; +END; +$$; + +SELECT fastpath_exceeded AS fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation' \gset + +-- Needs a lock on each partition +SELECT count(*) FROM part_test; + +-- Ensure pending stats are flushed +SELECT pg_stat_force_next_flush(); + +SELECT fastpath_exceeded > :fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation'; + +DROP TABLE part_test; + -- End of Stats Test -- 2.34.1 --nK2xgkLTeiiswcxd Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v9-0003-Introduce-a-new-track_lock_timing-GUC.patch" ^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v11 3/3] Add the pg_stat_lock view @ 2025-07-31 09:35 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 14+ messages in thread From: Bertrand Drouvot @ 2025-07-31 09:35 UTC (permalink / raw) This new view reports lock statistics. Note that it does not omit combinations which do not make sense (as pg_locks does). Also wait_time is reported as bigint as the deadlock_timeout default value is 1s. This commit also adds documentation and a few tests. XXX: Bump catversion Author: Bertrand Drouvot <[email protected]> Reviewed-by: Andres Freund <[email protected]> Discussion: https://postgr.es/m/aIyNxBWFCybgBZBS%40ip-10-97-1-34.eu-west-3.compute.internal --- doc/src/sgml/monitoring.sgml | 111 ++++++++++++ src/backend/catalog/system_views.sql | 9 + src/backend/utils/activity/pgstat_lock.c | 8 + src/backend/utils/adt/pgstatfuncs.c | 39 +++++ src/include/catalog/pg_proc.dat | 9 + src/include/pgstat.h | 1 + src/test/isolation/expected/lock-stats.out | 190 +++++++++++++++++++++ src/test/isolation/isolation_schedule | 1 + src/test/isolation/specs/lock-stats.spec | 125 ++++++++++++++ src/test/regress/expected/rules.out | 6 + src/test/regress/expected/stats.out | 48 ++++++ src/test/regress/sql/stats.sql | 36 ++++ 12 files changed, 583 insertions(+) 19.9% doc/src/sgml/ 6.4% src/backend/utils/adt/ 35.1% src/test/isolation/expected/ 20.8% src/test/isolation/specs/ 7.4% src/test/regress/expected/ 5.2% src/test/regress/sql/ 4.8% src/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 462019a972c..d590ecad541 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -509,6 +509,15 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser </entry> </row> + <row> + <entry><structname>pg_stat_lock</structname><indexterm><primary>pg_stat_lock</primary></indexterm></entry> + <entry> + One row for each lock type, containing cluster-wide locks statistics. + See <link linkend="monitoring-pg-stat-lock-view"> + <structname>pg_stat_lock</structname></link> for details. + </entry> + </row> + <row> <entry><structname>pg_stat_replication_slots</structname><indexterm><primary>pg_stat_replication_slots</primary></indexterm></entry> <entry>One row per replication slot, showing statistics about the @@ -3283,6 +3292,102 @@ description | Waiting for a newly initialized WAL file to reach durable storage </sect2> + + <sect2 id="monitoring-pg-stat-lock-view"> + <title><structname>pg_stat_lock</structname></title> + + <indexterm> + <primary>pg_stat_lock</primary> + </indexterm> + + <para> + The <structname>pg_stat_lock</structname> view will contain one row for each + lock type, showing cluster-wide locks statistics. + </para> + + <table id="pg-stat-lock-view" xreflabel="pg_stat_lock"> + <title><structname>pg_stat_lock</structname> View</title> + <tgroup cols="1"> + <thead> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + Column Type + </para> + <para> + Description + </para> + </entry> + </row> + </thead> + <tbody> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>locktype</structfield> <type>text</type> + </para> + <para> + Type of the lockable object. See <link linkend="view-pg-locks"> + <structname>pg_locks</structname></link> for details. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>waits</structfield> <type>bigint</type> + </para> + <para> + Number of times a lock of this type had to wait because of a + conflicting lock. Only incremented when the lock was successfully + acquired after waiting longer than <xref linkend="guc-deadlock-timeout"/>. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>wait_time</structfield> <type>bigint</type> + </para> + <para> + Total time spent waiting for locks of this type, in milliseconds. + Only incremented when the lock was successfully acquired after waiting + longer than <xref linkend="guc-deadlock-timeout"/>. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>fastpath_exceeded</structfield> <type>bigint</type> + </para> + <para> + Number of times a lock of this type could not be acquired via fast path + because the fast path slot limit was exceeded. You may want to increase + <xref linkend="guc-max-locks-per-transaction"/> if you feel this counter + is too high. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>stats_reset</structfield> <type>timestamp with time zone</type> + </para> + <para> + Time at which these statistics were last reset. + </para> + </entry> + </row> + </tbody> + </tgroup> + </table> + </sect2> + <sect2 id="monitoring-pg-stat-bgwriter-view"> <title><structname>pg_stat_bgwriter</structname></title> @@ -5372,6 +5477,12 @@ description | Waiting for a newly initialized WAL file to reach durable storage <structname>pg_stat_io</structname> view. </para> </listitem> + <listitem> + <para> + <literal>lock</literal>: Reset all the counters shown in the + <structname>pg_stat_lock</structname> view. + </para> + </listitem> <listitem> <para> <literal>recovery_prefetch</literal>: Reset all the counters shown in diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index f1ed7b58f13..e54018004db 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -985,6 +985,15 @@ CREATE VIEW pg_stat_slru AS s.stats_reset FROM pg_stat_get_slru() s; +CREATE VIEW pg_stat_lock AS + SELECT + l.locktype, + l.waits, + l.wait_time, + l.fastpath_exceeded, + l.stats_reset + FROM pg_stat_get_lock() l; + CREATE VIEW pg_stat_wal_receiver AS SELECT s.pid, diff --git a/src/backend/utils/activity/pgstat_lock.c b/src/backend/utils/activity/pgstat_lock.c index fe330eb99b1..76749e4724d 100644 --- a/src/backend/utils/activity/pgstat_lock.c +++ b/src/backend/utils/activity/pgstat_lock.c @@ -21,6 +21,14 @@ static PgStat_PendingLock PendingLockStats; static bool have_lockstats = false; +PgStat_Lock * +pgstat_fetch_stat_lock(void) +{ + pgstat_snapshot_fixed(PGSTAT_KIND_LOCK); + + return &pgStatLocal.snapshot.lock; +} + /* * Simpler wrapper of pgstat_lock_flush_cb() */ diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 5f907335990..9185a8e6b83 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1741,6 +1741,42 @@ pg_stat_get_wal(PG_FUNCTION_ARGS) wal_stats->stat_reset_timestamp)); } +Datum +pg_stat_get_lock(PG_FUNCTION_ARGS) +{ +#define PG_STAT_LOCK_COLS 5 + ReturnSetInfo *rsinfo; + PgStat_Lock *lock_stats; + + InitMaterializedSRF(fcinfo, 0); + rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; + + lock_stats = pgstat_fetch_stat_lock(); + + for (int lcktype = 0; lcktype <= LOCKTAG_LAST_TYPE; lcktype++) + { + const char *locktypename; + Datum values[PG_STAT_LOCK_COLS] = {0}; + bool nulls[PG_STAT_LOCK_COLS] = {0}; + PgStat_LockEntry *lck_stats = &lock_stats->stats[lcktype]; + int i = 0; + + locktypename = LockTagTypeNames[lcktype]; + + values[i++] = CStringGetTextDatum(locktypename); + values[i++] = Int64GetDatum(lck_stats->waits); + values[i++] = Int64GetDatum(lck_stats->wait_time); + values[i++] = Int64GetDatum(lck_stats->fastpath_exceeded); + values[i] = TimestampTzGetDatum(lock_stats->stat_reset_timestamp); + + Assert(i + 1 == PG_STAT_LOCK_COLS); + + tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls); + } + + return (Datum) 0; +} + /* * Returns statistics of SLRU caches. */ @@ -1925,6 +1961,7 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) pgstat_reset_of_kind(PGSTAT_KIND_BGWRITER); pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER); pgstat_reset_of_kind(PGSTAT_KIND_IO); + pgstat_reset_of_kind(PGSTAT_KIND_LOCK); XLogPrefetchResetStats(); pgstat_reset_of_kind(PGSTAT_KIND_SLRU); pgstat_reset_of_kind(PGSTAT_KIND_WAL); @@ -1942,6 +1979,8 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER); else if (strcmp(target, "io") == 0) pgstat_reset_of_kind(PGSTAT_KIND_IO); + else if (strcmp(target, "lock") == 0) + pgstat_reset_of_kind(PGSTAT_KIND_LOCK); else if (strcmp(target, "recovery_prefetch") == 0) XLogPrefetchResetStats(); else if (strcmp(target, "slru") == 0) diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index fc8d82665b8..3e2c43e9d1b 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6041,6 +6041,15 @@ proargnames => '{backend_type,object,context,reads,read_bytes,read_time,writes,write_bytes,write_time,writebacks,writeback_time,extends,extend_bytes,extend_time,hits,evictions,reuses,fsyncs,fsync_time,stats_reset}', prosrc => 'pg_stat_get_io' }, +{ oid => '9375', descr => 'statistics: per lock type statistics', + proname => 'pg_stat_get_lock', prorows => '10', proretset => 't', + provolatile => 'v', proparallel => 'r', prorettype => 'record', + proargtypes => '', + proallargtypes => '{text,int8,int8,int8,timestamptz}', + proargmodes => '{o,o,o,o,o}', + proargnames => '{locktype,waits,wait_time,fastpath_exceeded,stats_reset}', + prosrc => 'pg_stat_get_lock' }, + { oid => '6386', descr => 'statistics: backend IO statistics', proname => 'pg_stat_get_backend_io', prorows => '5', proretset => 't', provolatile => 'v', proparallel => 'r', prorettype => 'record', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 553106025da..dadf8b1c1ed 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -643,6 +643,7 @@ extern bool pgstat_tracks_io_op(BackendType bktype, IOObject io_object, extern void pgstat_lock_flush(bool nowait); extern void pgstat_count_lock_fastpath_exceeded(uint8 locktag_type); extern void pgstat_count_lock_waits(uint8 locktag_type, long msecs); +extern PgStat_Lock *pgstat_fetch_stat_lock(void); /* * Functions in pgstat_database.c diff --git a/src/test/isolation/expected/lock-stats.out b/src/test/isolation/expected/lock-stats.out new file mode 100644 index 00000000000..556c210a28b --- /dev/null +++ b/src/test/isolation/expected/lock-stats.out @@ -0,0 +1,190 @@ +Parsed test spec with 2 sessions + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_begin s1_lock_relation s2_begin s2_ff s2_lock_relation s1_sleep s1_commit s2_commit s2_report_stat_lock_relation +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_begin: BEGIN; +step s1_lock_relation: LOCK TABLE test_stat_tab; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_relation: LOCK TABLE test_stat_tab; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.05); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_lock_relation: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_relation: SELECT waits > 0, wait_time > 50 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column? +--------+-------- +t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_table_insert s1_begin s1_table_update_k1 s2_begin s2_ff s2_table_update_k1 s1_sleep s1_commit s2_commit s2_report_stat_lock_transactionid +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_table_insert: INSERT INTO test_stat_tab(key, value) VALUES('k1', 1), ('k2', 1), ('k3', 1); +step s1_begin: BEGIN; +step s1_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.05); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_table_update_k1: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_transactionid: SELECT waits > 0, wait_time > 50 FROM pg_stat_lock WHERE locktype = 'transactionid'; +?column?|?column? +--------+-------- +t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_lock_advisory_lock s2_begin s2_ff s2_lock_advisory_lock s1_sleep s1_lock_advisory_unlock s2_lock_advisory_unlock s2_commit s2_report_stat_lock_advisory +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_lock_advisory_lock: SELECT pg_advisory_lock(1); +pg_advisory_lock +---------------- + +(1 row) + +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_advisory_lock: SELECT pg_advisory_lock(1); <waiting ...> +step s1_sleep: SELECT pg_sleep(0.05); +pg_sleep +-------- + +(1 row) + +step s1_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_lock_advisory_lock: <... completed> +pg_advisory_lock +---------------- + +(1 row) + +step s2_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_commit: COMMIT; +step s2_report_stat_lock_advisory: SELECT waits > 0, wait_time > 50 FROM pg_stat_lock WHERE locktype = 'advisory'; +?column?|?column? +--------+-------- +t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_unset_log_lock_waits s1_begin s1_lock_relation s2_begin s2_ff s2_lock_relation s1_sleep s1_commit s2_commit s2_report_stat_lock_relation +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_unset_log_lock_waits: SET log_lock_waits = off; +step s1_begin: BEGIN; +step s1_lock_relation: LOCK TABLE test_stat_tab; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_relation: LOCK TABLE test_stat_tab; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.05); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_lock_relation: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_relation: SELECT waits > 0, wait_time > 50 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column? +--------+-------- +t |t +(1 row) + diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule index 4e466580cd4..2cc86849c9a 100644 --- a/src/test/isolation/isolation_schedule +++ b/src/test/isolation/isolation_schedule @@ -43,6 +43,7 @@ test: eval-plan-qual-trigger test: inplace-inval test: intra-grant-inplace test: intra-grant-inplace-db +test: lock-stats test: lock-update-delete test: lock-update-traversal test: inherit-temp diff --git a/src/test/isolation/specs/lock-stats.spec b/src/test/isolation/specs/lock-stats.spec new file mode 100644 index 00000000000..e4e66a9fb4d --- /dev/null +++ b/src/test/isolation/specs/lock-stats.spec @@ -0,0 +1,125 @@ +# Test for the lock statistics +# +# This test creates multiple locking situations when a session (s2) has to +# wait on a lock for longer than the deadlock timeout. The first permutations +# each test a dedicated lock type. The last permutation ensures that +# log_lock_waits has no impact on the statistics counters. + +setup +{ + CREATE TABLE test_stat_tab(key text not null, value int); + INSERT INTO test_stat_tab(key, value) VALUES('k0', 1); + SELECT pg_stat_force_next_flush(); +} + +teardown +{ + DROP TABLE IF EXISTS test_stat_tab; +} + +session s1 +setup { SET stats_fetch_consistency = 'none'; } +step s1_begin { BEGIN; } +step s1_commit { COMMIT; } +step s1_table_insert { INSERT INTO test_stat_tab(key, value) VALUES('k1', 1), ('k2', 1), ('k3', 1);} +step s1_table_update_k1 { UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1';} +step s1_set_deadlock_timeout { SET deadlock_timeout = '10ms'; } +step s1_set_log_lock_waits { SET log_lock_waits = on; } +step s1_reset_stat_lock { SELECT pg_stat_reset_shared('lock'); } +step s1_sleep { SELECT pg_sleep(0.05); } +step s1_lock_relation { LOCK TABLE test_stat_tab; } +step s1_lock_advisory_lock { SELECT pg_advisory_lock(1); } +step s1_lock_advisory_unlock { SELECT pg_advisory_unlock(1); } + +session s2 +setup { SET stats_fetch_consistency = 'none'; } +step s2_begin { BEGIN; } +step s2_commit { COMMIT; } +step s2_ff { SELECT pg_stat_force_next_flush(); } +step s2_table_update_k1 { UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1';} +step s2_set_deadlock_timeout { SET deadlock_timeout = '10ms'; } +step s2_set_log_lock_waits { SET log_lock_waits = on; } +step s2_unset_log_lock_waits { SET log_lock_waits = off; } +step s2_report_stat_lock_relation { SELECT waits > 0, wait_time > 50 FROM pg_stat_lock WHERE locktype = 'relation'; } +step s2_report_stat_lock_transactionid { SELECT waits > 0, wait_time > 50 FROM pg_stat_lock WHERE locktype = 'transactionid'; } +step s2_report_stat_lock_advisory { SELECT waits > 0, wait_time > 50 FROM pg_stat_lock WHERE locktype = 'advisory'; } +step s2_lock_relation { LOCK TABLE test_stat_tab; } +step s2_lock_advisory_lock { SELECT pg_advisory_lock(1); } +step s2_lock_advisory_unlock { SELECT pg_advisory_unlock(1); } + +###################### +# Lock stats tests +###################### + +# relation lock + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_set_log_lock_waits + s1_begin + s1_lock_relation + s2_begin + s2_ff + s2_lock_relation + s1_sleep + s1_commit + s2_commit + s2_report_stat_lock_relation + +# transaction lock + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_set_log_lock_waits + s1_table_insert + s1_begin + s1_table_update_k1 + s2_begin + s2_ff + s2_table_update_k1 + s1_sleep + s1_commit + s2_commit + s2_report_stat_lock_transactionid + +# advisory lock + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_set_log_lock_waits + s1_lock_advisory_lock + s2_begin + s2_ff + s2_lock_advisory_lock + s1_sleep + s1_lock_advisory_unlock + s2_lock_advisory_unlock + s2_commit + s2_report_stat_lock_advisory + +# Ensure log_lock_waits has no impact + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_unset_log_lock_waits + s1_begin + s1_lock_relation + s2_begin + s2_ff + s2_lock_relation + s1_sleep + s1_commit + s2_commit + s2_report_stat_lock_relation diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index 32bea58db2c..2b3cf6d8569 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1952,6 +1952,12 @@ pg_stat_io| SELECT backend_type, fsync_time, stats_reset FROM pg_stat_get_io() b(backend_type, object, context, reads, read_bytes, read_time, writes, write_bytes, write_time, writebacks, writeback_time, extends, extend_bytes, extend_time, hits, evictions, reuses, fsyncs, fsync_time, stats_reset); +pg_stat_lock| SELECT locktype, + waits, + wait_time, + fastpath_exceeded, + stats_reset + FROM pg_stat_get_lock() l(locktype, waits, wait_time, fastpath_exceeded, stats_reset); pg_stat_progress_analyze| SELECT s.pid, s.datid, d.datname, diff --git a/src/test/regress/expected/stats.out b/src/test/regress/expected/stats.out index b99462bf946..ea7f7846895 100644 --- a/src/test/regress/expected/stats.out +++ b/src/test/regress/expected/stats.out @@ -1958,4 +1958,52 @@ SELECT * FROM check_estimated_rows('SELECT * FROM table_fillfactor'); (1 row) DROP TABLE table_fillfactor; +-- Test fastpath_exceeded stat +CREATE TABLE part_test (id int) PARTITION BY RANGE (id); +SELECT pg_stat_reset_shared('lock'); + pg_stat_reset_shared +---------------------- + +(1 row) + +-- Create partitions (exceeds number of slots) +DO $$ +DECLARE + max_locks int; +BEGIN + SELECT setting::int INTO max_locks + FROM pg_settings + WHERE name = 'max_locks_per_transaction'; + + FOR i IN 1..(max_locks + 10) LOOP + EXECUTE format( + 'CREATE TABLE part_test_%s PARTITION OF part_test + FOR VALUES FROM (%s) TO (%s)', + i, (i-1)*1000, i*1000 + ); + END LOOP; +END; +$$; +SELECT fastpath_exceeded AS fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation' \gset +-- Needs a lock on each partition +SELECT count(*) FROM part_test; + count +------- + 0 +(1 row) + +-- Ensure pending stats are flushed +SELECT pg_stat_force_next_flush(); + pg_stat_force_next_flush +-------------------------- + +(1 row) + +SELECT fastpath_exceeded > :fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation'; + ?column? +---------- + t +(1 row) + +DROP TABLE part_test; -- End of Stats Test diff --git a/src/test/regress/sql/stats.sql b/src/test/regress/sql/stats.sql index 941222cf0be..65d8968c83e 100644 --- a/src/test/regress/sql/stats.sql +++ b/src/test/regress/sql/stats.sql @@ -964,4 +964,40 @@ SELECT * FROM check_estimated_rows('SELECT * FROM table_fillfactor'); DROP TABLE table_fillfactor; +-- Test fastpath_exceeded stat +CREATE TABLE part_test (id int) PARTITION BY RANGE (id); + +SELECT pg_stat_reset_shared('lock'); + +-- Create partitions (exceeds number of slots) +DO $$ +DECLARE + max_locks int; +BEGIN + SELECT setting::int INTO max_locks + FROM pg_settings + WHERE name = 'max_locks_per_transaction'; + + FOR i IN 1..(max_locks + 10) LOOP + EXECUTE format( + 'CREATE TABLE part_test_%s PARTITION OF part_test + FOR VALUES FROM (%s) TO (%s)', + i, (i-1)*1000, i*1000 + ); + END LOOP; +END; +$$; + +SELECT fastpath_exceeded AS fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation' \gset + +-- Needs a lock on each partition +SELECT count(*) FROM part_test; + +-- Ensure pending stats are flushed +SELECT pg_stat_force_next_flush(); + +SELECT fastpath_exceeded > :fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation'; + +DROP TABLE part_test; + -- End of Stats Test -- 2.34.1 --YVZ2rdHTqHRFZ4HX-- ^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v10 2/2] Add the pg_stat_lock view @ 2025-07-31 09:35 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 14+ messages in thread From: Bertrand Drouvot @ 2025-07-31 09:35 UTC (permalink / raw) This new view reports lock statistics. Note that it does not omit combinations which do not make sense (as pg_locks does). Also wait_time is reported as bigint as the deadlock_timeout default value is 1s. This commit also adds documentation and a few tests. XXX: Bump catversion Author: Bertrand Drouvot <[email protected]> Reviewed-by: Andres Freund <[email protected]> Discussion: https://postgr.es/m/aIyNxBWFCybgBZBS%40ip-10-97-1-34.eu-west-3.compute.internal --- doc/src/sgml/monitoring.sgml | 113 ++++++++++++ src/backend/catalog/system_views.sql | 9 + src/backend/utils/activity/pgstat_lock.c | 8 + src/backend/utils/adt/pgstatfuncs.c | 39 +++++ src/include/catalog/pg_proc.dat | 9 + src/include/pgstat.h | 1 + src/test/isolation/expected/lock-stats.out | 190 +++++++++++++++++++++ src/test/isolation/isolation_schedule | 1 + src/test/isolation/specs/lock-stats.spec | 118 +++++++++++++ src/test/regress/expected/rules.out | 6 + src/test/regress/expected/stats.out | 48 ++++++ src/test/regress/sql/stats.sql | 36 ++++ 12 files changed, 578 insertions(+) 20.8% doc/src/sgml/ 6.5% src/backend/utils/adt/ 35.5% src/test/isolation/expected/ 19.1% src/test/isolation/specs/ 7.5% src/test/regress/expected/ 5.3% src/test/regress/sql/ 4.9% src/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 462019a972c..221ee28cc7b 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -509,6 +509,15 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser </entry> </row> + <row> + <entry><structname>pg_stat_lock</structname><indexterm><primary>pg_stat_lock</primary></indexterm></entry> + <entry> + One row for each lock type, containing cluster-wide locks statistics. + See <link linkend="monitoring-pg-stat-lock-view"> + <structname>pg_stat_lock</structname></link> for details. + </entry> + </row> + <row> <entry><structname>pg_stat_replication_slots</structname><indexterm><primary>pg_stat_replication_slots</primary></indexterm></entry> <entry>One row per replication slot, showing statistics about the @@ -3283,6 +3292,104 @@ description | Waiting for a newly initialized WAL file to reach durable storage </sect2> + + <sect2 id="monitoring-pg-stat-lock-view"> + <title><structname>pg_stat_lock</structname></title> + + <indexterm> + <primary>pg_stat_lock</primary> + </indexterm> + + <para> + The <structname>pg_stat_lock</structname> view will contain one row for each + lock type, showing cluster-wide locks statistics. + </para> + + <table id="pg-stat-lock-view" xreflabel="pg_stat_lock"> + <title><structname>pg_stat_lock</structname> View</title> + <tgroup cols="1"> + <thead> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + Column Type + </para> + <para> + Description + </para> + </entry> + </row> + </thead> + <tbody> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>locktype</structfield> <type>text</type> + </para> + <para> + Type of the lockable object. See <link linkend="view-pg-locks"> + <structname>pg_locks</structname></link> for details. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>waits</structfield> <type>bigint</type> + </para> + <para> + Number of times a lock of this type had to wait because of a + conflicting lock. Only incremented when <xref linkend="guc-log-lock-waits"/> + is enabled and the lock was successfully acquired after waiting longer + than <xref linkend="guc-deadlock-timeout"/>. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>wait_time</structfield> <type>bigint</type> + </para> + <para> + Total time spent waiting for locks of this type, in milliseconds. + Only incremented when <xref linkend="guc-log-lock-waits"/> is enabled and + the lock was successfully acquired after waiting longer than + <xref linkend="guc-deadlock-timeout"/>. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>fastpath_exceeded</structfield> <type>bigint</type> + </para> + <para> + Number of times a lock of this type could not be acquired via fast path + because the fast path slot limit was exceeded. You may want to increase + <xref linkend="guc-max-locks-per-transaction"/> if you feel this counter + is too high. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>stats_reset</structfield> <type>timestamp with time zone</type> + </para> + <para> + Time at which these statistics were last reset. + </para> + </entry> + </row> + </tbody> + </tgroup> + </table> + </sect2> + <sect2 id="monitoring-pg-stat-bgwriter-view"> <title><structname>pg_stat_bgwriter</structname></title> @@ -5372,6 +5479,12 @@ description | Waiting for a newly initialized WAL file to reach durable storage <structname>pg_stat_io</structname> view. </para> </listitem> + <listitem> + <para> + <literal>lock</literal>: Reset all the counters shown in the + <structname>pg_stat_lock</structname> view. + </para> + </listitem> <listitem> <para> <literal>recovery_prefetch</literal>: Reset all the counters shown in diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index f1ed7b58f13..e54018004db 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -985,6 +985,15 @@ CREATE VIEW pg_stat_slru AS s.stats_reset FROM pg_stat_get_slru() s; +CREATE VIEW pg_stat_lock AS + SELECT + l.locktype, + l.waits, + l.wait_time, + l.fastpath_exceeded, + l.stats_reset + FROM pg_stat_get_lock() l; + CREATE VIEW pg_stat_wal_receiver AS SELECT s.pid, diff --git a/src/backend/utils/activity/pgstat_lock.c b/src/backend/utils/activity/pgstat_lock.c index 9ec61f2b2ec..eccb12b0b68 100644 --- a/src/backend/utils/activity/pgstat_lock.c +++ b/src/backend/utils/activity/pgstat_lock.c @@ -21,6 +21,14 @@ static PgStat_PendingLock PendingLockStats; static bool have_lockstats = false; +PgStat_Lock * +pgstat_fetch_stat_lock(void) +{ + pgstat_snapshot_fixed(PGSTAT_KIND_LOCK); + + return &pgStatLocal.snapshot.lock; +} + /* * Simpler wrapper of pgstat_lock_flush_cb() */ diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 5f907335990..9185a8e6b83 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1741,6 +1741,42 @@ pg_stat_get_wal(PG_FUNCTION_ARGS) wal_stats->stat_reset_timestamp)); } +Datum +pg_stat_get_lock(PG_FUNCTION_ARGS) +{ +#define PG_STAT_LOCK_COLS 5 + ReturnSetInfo *rsinfo; + PgStat_Lock *lock_stats; + + InitMaterializedSRF(fcinfo, 0); + rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; + + lock_stats = pgstat_fetch_stat_lock(); + + for (int lcktype = 0; lcktype <= LOCKTAG_LAST_TYPE; lcktype++) + { + const char *locktypename; + Datum values[PG_STAT_LOCK_COLS] = {0}; + bool nulls[PG_STAT_LOCK_COLS] = {0}; + PgStat_LockEntry *lck_stats = &lock_stats->stats[lcktype]; + int i = 0; + + locktypename = LockTagTypeNames[lcktype]; + + values[i++] = CStringGetTextDatum(locktypename); + values[i++] = Int64GetDatum(lck_stats->waits); + values[i++] = Int64GetDatum(lck_stats->wait_time); + values[i++] = Int64GetDatum(lck_stats->fastpath_exceeded); + values[i] = TimestampTzGetDatum(lock_stats->stat_reset_timestamp); + + Assert(i + 1 == PG_STAT_LOCK_COLS); + + tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls); + } + + return (Datum) 0; +} + /* * Returns statistics of SLRU caches. */ @@ -1925,6 +1961,7 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) pgstat_reset_of_kind(PGSTAT_KIND_BGWRITER); pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER); pgstat_reset_of_kind(PGSTAT_KIND_IO); + pgstat_reset_of_kind(PGSTAT_KIND_LOCK); XLogPrefetchResetStats(); pgstat_reset_of_kind(PGSTAT_KIND_SLRU); pgstat_reset_of_kind(PGSTAT_KIND_WAL); @@ -1942,6 +1979,8 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER); else if (strcmp(target, "io") == 0) pgstat_reset_of_kind(PGSTAT_KIND_IO); + else if (strcmp(target, "lock") == 0) + pgstat_reset_of_kind(PGSTAT_KIND_LOCK); else if (strcmp(target, "recovery_prefetch") == 0) XLogPrefetchResetStats(); else if (strcmp(target, "slru") == 0) diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index fc8d82665b8..3e2c43e9d1b 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6041,6 +6041,15 @@ proargnames => '{backend_type,object,context,reads,read_bytes,read_time,writes,write_bytes,write_time,writebacks,writeback_time,extends,extend_bytes,extend_time,hits,evictions,reuses,fsyncs,fsync_time,stats_reset}', prosrc => 'pg_stat_get_io' }, +{ oid => '9375', descr => 'statistics: per lock type statistics', + proname => 'pg_stat_get_lock', prorows => '10', proretset => 't', + provolatile => 'v', proparallel => 'r', prorettype => 'record', + proargtypes => '', + proallargtypes => '{text,int8,int8,int8,timestamptz}', + proargmodes => '{o,o,o,o,o}', + proargnames => '{locktype,waits,wait_time,fastpath_exceeded,stats_reset}', + prosrc => 'pg_stat_get_lock' }, + { oid => '6386', descr => 'statistics: backend IO statistics', proname => 'pg_stat_get_backend_io', prorows => '5', proretset => 't', provolatile => 'v', proparallel => 'r', prorettype => 'record', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 4a28ea99705..06e2638e788 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -643,6 +643,7 @@ extern bool pgstat_tracks_io_op(BackendType bktype, IOObject io_object, extern void pgstat_lock_flush(bool nowait); extern void pgstat_count_lock_fastpath_exceeded(uint8 locktag_type); extern void pgstat_count_lock_waits(uint8 locktag_type, long msecs); +extern PgStat_Lock *pgstat_fetch_stat_lock(void); /* * Functions in pgstat_database.c diff --git a/src/test/isolation/expected/lock-stats.out b/src/test/isolation/expected/lock-stats.out new file mode 100644 index 00000000000..fa9e141dc20 --- /dev/null +++ b/src/test/isolation/expected/lock-stats.out @@ -0,0 +1,190 @@ +Parsed test spec with 2 sessions + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_begin s1_lock_relation s2_begin s2_ff s2_lock_relation s1_sleep s1_commit s2_commit s2_report_stat_lock_relation +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_begin: BEGIN; +step s1_lock_relation: LOCK TABLE test_stat_tab; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_relation: LOCK TABLE test_stat_tab; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.05); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_lock_relation: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_relation: SELECT waits > 0, wait_time > 50 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column? +--------+-------- +t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_table_insert s1_begin s1_table_update_k1 s2_begin s2_ff s2_table_update_k1 s1_sleep s1_commit s2_commit s2_report_stat_lock_transactionid +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_table_insert: INSERT INTO test_stat_tab(key, value) VALUES('k1', 1), ('k2', 1), ('k3', 1); +step s1_begin: BEGIN; +step s1_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.05); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_table_update_k1: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_transactionid: SELECT waits > 0, wait_time > 50 FROM pg_stat_lock WHERE locktype = 'transactionid'; +?column?|?column? +--------+-------- +t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_set_log_lock_waits s1_lock_advisory_lock s2_begin s2_ff s2_lock_advisory_lock s1_sleep s1_lock_advisory_unlock s2_lock_advisory_unlock s2_commit s2_report_stat_lock_advisory +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_log_lock_waits: SET log_lock_waits = on; +step s1_lock_advisory_lock: SELECT pg_advisory_lock(1); +pg_advisory_lock +---------------- + +(1 row) + +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_advisory_lock: SELECT pg_advisory_lock(1); <waiting ...> +step s1_sleep: SELECT pg_sleep(0.05); +pg_sleep +-------- + +(1 row) + +step s1_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_lock_advisory_lock: <... completed> +pg_advisory_lock +---------------- + +(1 row) + +step s2_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_commit: COMMIT; +step s2_report_stat_lock_advisory: SELECT waits > 0, wait_time > 50 FROM pg_stat_lock WHERE locktype = 'advisory'; +?column?|?column? +--------+-------- +t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_log_lock_waits s2_set_deadlock_timeout s2_unset_log_lock_waits s1_begin s1_lock_relation s2_begin s2_ff s2_lock_relation s1_sleep s1_commit s2_commit s2_report_stat_lock_relation +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_log_lock_waits: SET log_lock_waits = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_unset_log_lock_waits: SET log_lock_waits = off; +step s1_begin: BEGIN; +step s1_lock_relation: LOCK TABLE test_stat_tab; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_relation: LOCK TABLE test_stat_tab; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.05); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_lock_relation: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_relation: SELECT waits > 0, wait_time > 50 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column? +--------+-------- +f |f +(1 row) + diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule index 4e466580cd4..2cc86849c9a 100644 --- a/src/test/isolation/isolation_schedule +++ b/src/test/isolation/isolation_schedule @@ -43,6 +43,7 @@ test: eval-plan-qual-trigger test: inplace-inval test: intra-grant-inplace test: intra-grant-inplace-db +test: lock-stats test: lock-update-delete test: lock-update-traversal test: inherit-temp diff --git a/src/test/isolation/specs/lock-stats.spec b/src/test/isolation/specs/lock-stats.spec new file mode 100644 index 00000000000..2c5c91791e9 --- /dev/null +++ b/src/test/isolation/specs/lock-stats.spec @@ -0,0 +1,118 @@ +setup +{ + CREATE TABLE test_stat_tab(key text not null, value int); + INSERT INTO test_stat_tab(key, value) VALUES('k0', 1); + SELECT pg_stat_force_next_flush(); +} + +teardown +{ + DROP TABLE IF EXISTS test_stat_tab; +} + +session s1 +setup { SET stats_fetch_consistency = 'none'; } +step s1_begin { BEGIN; } +step s1_commit { COMMIT; } +step s1_table_insert { INSERT INTO test_stat_tab(key, value) VALUES('k1', 1), ('k2', 1), ('k3', 1);} +step s1_table_update_k1 { UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1';} +step s1_set_deadlock_timeout { SET deadlock_timeout = '10ms'; } +step s1_set_log_lock_waits { SET log_lock_waits = on; } +step s1_reset_stat_lock { SELECT pg_stat_reset_shared('lock'); } +step s1_sleep { SELECT pg_sleep(0.05); } +step s1_lock_relation { LOCK TABLE test_stat_tab; } +step s1_lock_advisory_lock { SELECT pg_advisory_lock(1); } +step s1_lock_advisory_unlock { SELECT pg_advisory_unlock(1); } + +session s2 +setup { SET stats_fetch_consistency = 'none'; } +step s2_begin { BEGIN; } +step s2_commit { COMMIT; } +step s2_ff { SELECT pg_stat_force_next_flush(); } +step s2_table_update_k1 { UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1';} +step s2_set_deadlock_timeout { SET deadlock_timeout = '10ms'; } +step s2_set_log_lock_waits { SET log_lock_waits = on; } +step s2_unset_log_lock_waits { SET log_lock_waits = off; } +step s2_report_stat_lock_relation { SELECT waits > 0, wait_time > 50 FROM pg_stat_lock WHERE locktype = 'relation'; } +step s2_report_stat_lock_transactionid { SELECT waits > 0, wait_time > 50 FROM pg_stat_lock WHERE locktype = 'transactionid'; } +step s2_report_stat_lock_advisory { SELECT waits > 0, wait_time > 50 FROM pg_stat_lock WHERE locktype = 'advisory'; } +step s2_lock_relation { LOCK TABLE test_stat_tab; } +step s2_lock_advisory_lock { SELECT pg_advisory_lock(1); } +step s2_lock_advisory_unlock { SELECT pg_advisory_unlock(1); } + +###################### +# Lock stats tests +###################### + +# relation lock + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_set_log_lock_waits + s1_begin + s1_lock_relation + s2_begin + s2_ff + s2_lock_relation + s1_sleep + s1_commit + s2_commit + s2_report_stat_lock_relation + +# transaction lock + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_set_log_lock_waits + s1_table_insert + s1_begin + s1_table_update_k1 + s2_begin + s2_ff + s2_table_update_k1 + s1_sleep + s1_commit + s2_commit + s2_report_stat_lock_transactionid + +# advisory lock + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_set_log_lock_waits + s1_lock_advisory_lock + s2_begin + s2_ff + s2_lock_advisory_lock + s1_sleep + s1_lock_advisory_unlock + s2_lock_advisory_unlock + s2_commit + s2_report_stat_lock_advisory + +# Ensure log_lock_waits behaves correctly + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_log_lock_waits + s2_set_deadlock_timeout + s2_unset_log_lock_waits + s1_begin + s1_lock_relation + s2_begin + s2_ff + s2_lock_relation + s1_sleep + s1_commit + s2_commit + s2_report_stat_lock_relation diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index 32bea58db2c..2b3cf6d8569 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1952,6 +1952,12 @@ pg_stat_io| SELECT backend_type, fsync_time, stats_reset FROM pg_stat_get_io() b(backend_type, object, context, reads, read_bytes, read_time, writes, write_bytes, write_time, writebacks, writeback_time, extends, extend_bytes, extend_time, hits, evictions, reuses, fsyncs, fsync_time, stats_reset); +pg_stat_lock| SELECT locktype, + waits, + wait_time, + fastpath_exceeded, + stats_reset + FROM pg_stat_get_lock() l(locktype, waits, wait_time, fastpath_exceeded, stats_reset); pg_stat_progress_analyze| SELECT s.pid, s.datid, d.datname, diff --git a/src/test/regress/expected/stats.out b/src/test/regress/expected/stats.out index b99462bf946..ea7f7846895 100644 --- a/src/test/regress/expected/stats.out +++ b/src/test/regress/expected/stats.out @@ -1958,4 +1958,52 @@ SELECT * FROM check_estimated_rows('SELECT * FROM table_fillfactor'); (1 row) DROP TABLE table_fillfactor; +-- Test fastpath_exceeded stat +CREATE TABLE part_test (id int) PARTITION BY RANGE (id); +SELECT pg_stat_reset_shared('lock'); + pg_stat_reset_shared +---------------------- + +(1 row) + +-- Create partitions (exceeds number of slots) +DO $$ +DECLARE + max_locks int; +BEGIN + SELECT setting::int INTO max_locks + FROM pg_settings + WHERE name = 'max_locks_per_transaction'; + + FOR i IN 1..(max_locks + 10) LOOP + EXECUTE format( + 'CREATE TABLE part_test_%s PARTITION OF part_test + FOR VALUES FROM (%s) TO (%s)', + i, (i-1)*1000, i*1000 + ); + END LOOP; +END; +$$; +SELECT fastpath_exceeded AS fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation' \gset +-- Needs a lock on each partition +SELECT count(*) FROM part_test; + count +------- + 0 +(1 row) + +-- Ensure pending stats are flushed +SELECT pg_stat_force_next_flush(); + pg_stat_force_next_flush +-------------------------- + +(1 row) + +SELECT fastpath_exceeded > :fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation'; + ?column? +---------- + t +(1 row) + +DROP TABLE part_test; -- End of Stats Test diff --git a/src/test/regress/sql/stats.sql b/src/test/regress/sql/stats.sql index 941222cf0be..65d8968c83e 100644 --- a/src/test/regress/sql/stats.sql +++ b/src/test/regress/sql/stats.sql @@ -964,4 +964,40 @@ SELECT * FROM check_estimated_rows('SELECT * FROM table_fillfactor'); DROP TABLE table_fillfactor; +-- Test fastpath_exceeded stat +CREATE TABLE part_test (id int) PARTITION BY RANGE (id); + +SELECT pg_stat_reset_shared('lock'); + +-- Create partitions (exceeds number of slots) +DO $$ +DECLARE + max_locks int; +BEGIN + SELECT setting::int INTO max_locks + FROM pg_settings + WHERE name = 'max_locks_per_transaction'; + + FOR i IN 1..(max_locks + 10) LOOP + EXECUTE format( + 'CREATE TABLE part_test_%s PARTITION OF part_test + FOR VALUES FROM (%s) TO (%s)', + i, (i-1)*1000, i*1000 + ); + END LOOP; +END; +$$; + +SELECT fastpath_exceeded AS fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation' \gset + +-- Needs a lock on each partition +SELECT count(*) FROM part_test; + +-- Ensure pending stats are flushed +SELECT pg_stat_force_next_flush(); + +SELECT fastpath_exceeded > :fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation'; + +DROP TABLE part_test; + -- End of Stats Test -- 2.34.1 --NJ5Y8xm2S5PE3Q71-- ^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v1 2/2] Add the pg_stat_lock view @ 2025-07-31 09:35 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 14+ messages in thread From: Bertrand Drouvot @ 2025-07-31 09:35 UTC (permalink / raw) This new view reports lock statistics. This commit also adds documentation and a few tests. XXX: Bump catversion --- doc/src/sgml/monitoring.sgml | 151 ++++++++++++++++++ src/backend/catalog/system_views.sql | 12 ++ src/backend/utils/activity/pgstat_lock.c | 8 + src/backend/utils/adt/pgstatfuncs.c | 39 +++++ src/include/catalog/pg_proc.dat | 9 ++ src/include/pgstat.h | 1 + src/test/isolation/expected/deadlock-hard.out | 20 ++- src/test/isolation/specs/deadlock-hard.spec | 5 +- src/test/regress/expected/advisory_lock.out | 18 +++ src/test/regress/expected/rules.out | 9 ++ src/test/regress/sql/advisory_lock.sql | 4 + 11 files changed, 274 insertions(+), 2 deletions(-) 51.5% doc/src/sgml/ 3.3% src/backend/catalog/ 14.3% src/backend/utils/adt/ 5.7% src/include/catalog/ 8.4% src/test/isolation/expected/ 5.9% src/test/isolation/specs/ 6.7% src/test/regress/expected/ 3.8% src/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 823afe1b30b..0d18124f239 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -493,6 +493,15 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser </entry> </row> + <row> + <entry><structname>pg_stat_lock</structname><indexterm><primary>pg_stat_lock</primary></indexterm></entry> + <entry> + One row for each lock type, containing cluster-wide locks statistics. + See <link linkend="monitoring-pg-stat-lock-view"> + <structname>pg_stat_lock</structname></link> for details. + </entry> + </row> + <row> <entry><structname>pg_stat_replication_slots</structname><indexterm><primary>pg_stat_replication_slots</primary></indexterm></entry> <entry>One row per replication slot, showing statistics about the @@ -3027,6 +3036,142 @@ description | Waiting for a newly initialized WAL file to reach durable storage </sect2> + + <sect2 id="monitoring-pg-stat-lock-view"> + <title><structname>pg_stat_lock</structname></title> + + <indexterm> + <primary>pg_stat_lock</primary> + </indexterm> + + <para> + The <structname>pg_stat_lock</structname> view will contain one row for each + lock type, showing cluster-wide locks statistics. + </para> + + <table id="pg-stat-lock-view" xreflabel="pg_stat_lock"> + <title><structname>pg_stat_lock</structname> View</title> + <tgroup cols="1"> + <thead> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + Column Type + </para> + <para> + Description + </para> + </entry> + </row> + </thead> + <tbody> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>locktype</structfield> <type>text</type> + </para> + <para> + Type of the lockable object: + <literal>relation</literal>, + <literal>extend</literal>, + <literal>frozenid</literal>, + <literal>page</literal>, + <literal>tuple</literal>, + <literal>transactionid</literal>, + <literal>virtualxid</literal>, + <literal>spectoken</literal>, + <literal>object</literal>, + <literal>userlock</literal>, + <literal>advisory</literal>, or + <literal>applytransaction</literal>. + (See also <xref linkend="wait-event-lock-table"/>.) + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>requests</structfield> <type>bigint</type> + </para> + <para> + Number of requests for this lock type. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>waits</structfield> <type>bigint</type> + </para> + <para> + Number of times requests for this lock type had to wait. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>timeouts</structfield> <type>bigint</type> + </para> + <para> + Number of times requests for this lock type had to wait longer + than <varname>lock_timeout</varname>. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>deadlock_timeouts</structfield> <type>bigint</type> + </para> + <para> + Number of times requests for this lock type had to wait longer + than <varname>deadlock_timeout</varname>. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>deadlocks</structfield> <type>bigint</type> + </para> + <para> + Number of times a deadlock occurred on this lock type. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>fastpath</structfield> <type>bigint</type> + </para> + <para> + Number of times this lock type was taken via fast path. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>stats_reset</structfield> <type>timestamp with time zone</type> + </para> + <para> + Time at which these statistics were last reset. + </para> + </entry> + </row> + </tbody> + </tgroup> + </table> + </sect2> + <sect2 id="monitoring-pg-stat-bgwriter-view"> <title><structname>pg_stat_bgwriter</structname></title> @@ -5044,6 +5189,12 @@ description | Waiting for a newly initialized WAL file to reach durable storage <structname>pg_stat_io</structname> view. </para> </listitem> + <listitem> + <para> + <literal>lock</literal>: Reset all the counters shown in the + <structname>pg_stat_lock</structname> view. + </para> + </listitem> <listitem> <para> <literal>recovery_prefetch</literal>: Reset all the counters shown in diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index f6eca09ee15..d493a00c544 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -950,6 +950,18 @@ CREATE VIEW pg_stat_slru AS s.stats_reset FROM pg_stat_get_slru() s; +CREATE VIEW pg_stat_lock AS + SELECT + l.locktype, + l.requests, + l.waits, + l.timeouts, + l.deadlock_timeouts, + l.deadlocks, + l.fastpath, + l.stats_reset + FROM pg_stat_get_lock() l; + CREATE VIEW pg_stat_wal_receiver AS SELECT s.pid, diff --git a/src/backend/utils/activity/pgstat_lock.c b/src/backend/utils/activity/pgstat_lock.c index fb4ed5c7362..73bdeb5ff66 100644 --- a/src/backend/utils/activity/pgstat_lock.c +++ b/src/backend/utils/activity/pgstat_lock.c @@ -21,6 +21,14 @@ static PgStat_PendingLock PendingLockStats; static bool have_lockstats = false; +PgStat_Lock * +pgstat_fetch_stat_lock(void) +{ + pgstat_snapshot_fixed(PGSTAT_KIND_LOCK); + + return &pgStatLocal.snapshot.lock; +} + /* * Simpler wrapper of pgstat_lock_flush_cb() */ diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 1c12ddbae49..5e63f2d2a38 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1696,6 +1696,42 @@ pg_stat_get_wal(PG_FUNCTION_ARGS) wal_stats->stat_reset_timestamp)); } +Datum +pg_stat_get_lock(PG_FUNCTION_ARGS) +{ +#define PG_STAT_LOCK_COLS 8 + ReturnSetInfo *rsinfo; + PgStat_Lock *lock_stats; + + InitMaterializedSRF(fcinfo, 0); + rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; + + lock_stats = pgstat_fetch_stat_lock(); + + for (int lcktype = 0; lcktype <= LOCKTAG_LAST_TYPE; lcktype++) + { + const char *locktypename; + Datum values[PG_STAT_LOCK_COLS] = {0}; + bool nulls[PG_STAT_LOCK_COLS] = {0}; + PgStat_LockEntry *lck_stats = &lock_stats->stats[lcktype]; + + locktypename = LockTagTypeNames[lcktype]; + + values[0] = CStringGetTextDatum(locktypename); + values[1] = Int64GetDatum(lck_stats->requests); + values[2] = Int64GetDatum(lck_stats->waits); + values[3] = Int64GetDatum(lck_stats->timeouts); + values[4] = Int64GetDatum(lck_stats->deadlock_timeouts); + values[5] = Int64GetDatum(lck_stats->deadlocks); + values[6] = Int64GetDatum(lck_stats->fastpath); + values[7] = TimestampTzGetDatum(lock_stats->stat_reset_timestamp); + + tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls); + } + + return (Datum) 0; +} + /* * Returns statistics of SLRU caches. */ @@ -1880,6 +1916,7 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) pgstat_reset_of_kind(PGSTAT_KIND_BGWRITER); pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER); pgstat_reset_of_kind(PGSTAT_KIND_IO); + pgstat_reset_of_kind(PGSTAT_KIND_LOCK); XLogPrefetchResetStats(); pgstat_reset_of_kind(PGSTAT_KIND_SLRU); pgstat_reset_of_kind(PGSTAT_KIND_WAL); @@ -1897,6 +1934,8 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER); else if (strcmp(target, "io") == 0) pgstat_reset_of_kind(PGSTAT_KIND_IO); + else if (strcmp(target, "lock") == 0) + pgstat_reset_of_kind(PGSTAT_KIND_LOCK); else if (strcmp(target, "recovery_prefetch") == 0) XLogPrefetchResetStats(); else if (strcmp(target, "slru") == 0) diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 3ee8fed7e53..c5faff30317 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6001,6 +6001,15 @@ proargnames => '{backend_type,object,context,reads,read_bytes,read_time,writes,write_bytes,write_time,writebacks,writeback_time,extends,extend_bytes,extend_time,hits,evictions,reuses,fsyncs,fsync_time,stats_reset}', prosrc => 'pg_stat_get_io' }, +{ oid => '9375', descr => 'statistics: per lock type statistics', + proname => 'pg_stat_get_lock', prorows => '10', proretset => 't', + provolatile => 'v', proparallel => 'r', prorettype => 'record', + proargtypes => '', + proallargtypes => '{text,int8,int8,int8,int8,int8,int8,timestamptz}', + proargmodes => '{o,o,o,o,o,o,o,o}', + proargnames => '{locktype,requests,waits,timeouts,deadlock_timeouts,deadlocks,fastpath,stats_reset}', + prosrc => 'pg_stat_get_lock' }, + { oid => '6386', descr => 'statistics: backend IO statistics', proname => 'pg_stat_get_backend_io', prorows => '5', proretset => 't', provolatile => 'v', proparallel => 'r', prorettype => 'record', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 7893f7311ae..e653bce4aab 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -636,6 +636,7 @@ extern void pgstat_count_lock_timeouts(uint8 locktag_type); extern void pgstat_count_lock_deadlock_timeouts(uint8 locktag_type); extern void pgstat_count_lock_deadlocks(uint8 locktag_type); extern void pgstat_count_lock_fastpath(uint8 locktag_type); +extern PgStat_Lock *pgstat_fetch_stat_lock(void); /* * Functions in pgstat_database.c diff --git a/src/test/isolation/expected/deadlock-hard.out b/src/test/isolation/expected/deadlock-hard.out index 460653f2b86..ff448da5299 100644 --- a/src/test/isolation/expected/deadlock-hard.out +++ b/src/test/isolation/expected/deadlock-hard.out @@ -1,6 +1,12 @@ Parsed test spec with 8 sessions -starting permutation: s1a1 s2a2 s3a3 s4a4 s5a5 s6a6 s7a7 s8a8 s1a2 s2a3 s3a4 s4a5 s5a6 s6a7 s7a8 s8a1 s8c s7c s6c s5c s4c s3c s2c s1c +starting permutation: s1rl s1a1 s2a2 s3a3 s4a4 s5a5 s6a6 s7a7 s8a8 s1a2 s2a3 s3a4 s4a5 s5a6 s6a7 s7a8 s8a1 s8c s8f s7c s6c s5c s4c s3c s2c s1c s1sl +step s1rl: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + step s1a1: LOCK TABLE a1; step s2a2: LOCK TABLE a2; step s3a3: LOCK TABLE a3; @@ -21,6 +27,12 @@ step s8a1: <... completed> ERROR: deadlock detected step s7a8: <... completed> step s8c: COMMIT; +step s8f: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + step s7c: COMMIT; step s6a7: <... completed> step s6c: COMMIT; @@ -34,3 +46,9 @@ step s2a3: <... completed> step s2c: COMMIT; step s1a2: <... completed> step s1c: COMMIT; +step s1sl: SELECT deadlocks > 0, deadlock_timeouts > 0 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column? +--------+-------- +t |t +(1 row) + diff --git a/src/test/isolation/specs/deadlock-hard.spec b/src/test/isolation/specs/deadlock-hard.spec index 60bedca237a..6e8330662b4 100644 --- a/src/test/isolation/specs/deadlock-hard.spec +++ b/src/test/isolation/specs/deadlock-hard.spec @@ -25,6 +25,8 @@ setup { BEGIN; SET deadlock_timeout = '100s'; } step s1a1 { LOCK TABLE a1; } step s1a2 { LOCK TABLE a2; } step s1c { COMMIT; } +step s1sl { SELECT deadlocks > 0, deadlock_timeouts > 0 FROM pg_stat_lock WHERE locktype = 'relation'; } +step s1rl { SELECT pg_stat_reset_shared('lock'); } session s2 setup { BEGIN; SET deadlock_timeout = '100s'; } @@ -67,6 +69,7 @@ setup { BEGIN; SET deadlock_timeout = '10ms'; } step s8a8 { LOCK TABLE a8; } step s8a1 { LOCK TABLE a1; } step s8c { COMMIT; } +step s8f { SELECT pg_stat_force_next_flush(); } # Note: when s8a1 detects the deadlock and fails, s7a8 is released, making # it timing-dependent which query completion is received first by the tester. @@ -76,4 +79,4 @@ step s8c { COMMIT; } # dummy blocking mark to s8a1 to ensure it will be reported as "waiting" # regardless of that. -permutation s1a1 s2a2 s3a3 s4a4 s5a5 s6a6 s7a7 s8a8 s1a2 s2a3 s3a4 s4a5 s5a6 s6a7 s7a8(s8a1) s8a1(*) s8c s7c s6c s5c s4c s3c s2c s1c +permutation s1rl s1a1 s2a2 s3a3 s4a4 s5a5 s6a6 s7a7 s8a8 s1a2 s2a3 s3a4 s4a5 s5a6 s6a7 s7a8(s8a1) s8a1(*) s8c s8f s7c s6c s5c s4c s3c s2c s1c s1sl diff --git a/src/test/regress/expected/advisory_lock.out b/src/test/regress/expected/advisory_lock.out index 02e07765ac2..fdaa1756ba4 100644 --- a/src/test/regress/expected/advisory_lock.out +++ b/src/test/regress/expected/advisory_lock.out @@ -2,6 +2,12 @@ -- ADVISORY LOCKS -- SELECT oid AS datoid FROM pg_database WHERE datname = current_database() \gset +SELECT pg_stat_reset_shared('lock'); + pg_stat_reset_shared +---------------------- + +(1 row) + BEGIN; SELECT pg_advisory_xact_lock(1), pg_advisory_xact_lock_shared(2), @@ -48,6 +54,12 @@ WARNING: you don't own a lock of type ShareLock f | f | f | f (1 row) +SELECT pg_stat_force_next_flush(); + pg_stat_force_next_flush +-------------------------- + +(1 row) + -- automatically release xact locks at commit COMMIT; SELECT count(*) FROM pg_locks WHERE locktype = 'advisory' AND database = :datoid; @@ -56,6 +68,12 @@ SELECT count(*) FROM pg_locks WHERE locktype = 'advisory' AND database = :datoid 0 (1 row) +SELECT requests FROM pg_stat_lock WHERE locktype = 'advisory'; + requests +---------- + 4 +(1 row) + BEGIN; -- holding both session and xact locks on the same objects, xact first SELECT diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index dce8c672b40..c790164ea82 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1938,6 +1938,15 @@ pg_stat_io| SELECT backend_type, fsync_time, stats_reset FROM pg_stat_get_io() b(backend_type, object, context, reads, read_bytes, read_time, writes, write_bytes, write_time, writebacks, writeback_time, extends, extend_bytes, extend_time, hits, evictions, reuses, fsyncs, fsync_time, stats_reset); +pg_stat_lock| SELECT locktype, + requests, + waits, + timeouts, + deadlock_timeouts, + deadlocks, + fastpath, + stats_reset + FROM pg_stat_get_lock() l(locktype, requests, waits, timeouts, deadlock_timeouts, deadlocks, fastpath, stats_reset); pg_stat_progress_analyze| SELECT s.pid, s.datid, d.datname, diff --git a/src/test/regress/sql/advisory_lock.sql b/src/test/regress/sql/advisory_lock.sql index 8513ab8e98f..f1bff60fd37 100644 --- a/src/test/regress/sql/advisory_lock.sql +++ b/src/test/regress/sql/advisory_lock.sql @@ -4,6 +4,8 @@ SELECT oid AS datoid FROM pg_database WHERE datname = current_database() \gset +SELECT pg_stat_reset_shared('lock'); + BEGIN; SELECT @@ -26,12 +28,14 @@ SELECT pg_advisory_unlock(1), pg_advisory_unlock_shared(2), pg_advisory_unlock(1, 1), pg_advisory_unlock_shared(2, 2); +SELECT pg_stat_force_next_flush(); -- automatically release xact locks at commit COMMIT; SELECT count(*) FROM pg_locks WHERE locktype = 'advisory' AND database = :datoid; +SELECT requests FROM pg_stat_lock WHERE locktype = 'advisory'; BEGIN; -- 2.34.1 --Th7cMauVQntl/Hn1-- ^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v2 2/2] Add the pg_stat_lock view @ 2025-07-31 09:35 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 14+ messages in thread From: Bertrand Drouvot @ 2025-07-31 09:35 UTC (permalink / raw) This new view reports lock statistics. This commit also adds documentation and a few tests. XXX: Bump catversion --- doc/src/sgml/monitoring.sgml | 151 ++++++++++++++++++ src/backend/catalog/system_views.sql | 12 ++ src/backend/utils/activity/pgstat_lock.c | 8 + src/backend/utils/adt/pgstatfuncs.c | 39 +++++ src/include/catalog/pg_proc.dat | 9 ++ src/include/pgstat.h | 1 + src/test/isolation/expected/deadlock-hard.out | 20 ++- src/test/isolation/specs/deadlock-hard.spec | 5 +- src/test/regress/expected/advisory_lock.out | 18 +++ src/test/regress/expected/rules.out | 9 ++ src/test/regress/sql/advisory_lock.sql | 4 + 11 files changed, 274 insertions(+), 2 deletions(-) 51.5% doc/src/sgml/ 3.3% src/backend/catalog/ 14.3% src/backend/utils/adt/ 5.7% src/include/catalog/ 8.4% src/test/isolation/expected/ 5.9% src/test/isolation/specs/ 6.7% src/test/regress/expected/ 3.8% src/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 817fd9f4ca7..3c327093543 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -493,6 +493,15 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser </entry> </row> + <row> + <entry><structname>pg_stat_lock</structname><indexterm><primary>pg_stat_lock</primary></indexterm></entry> + <entry> + One row for each lock type, containing cluster-wide locks statistics. + See <link linkend="monitoring-pg-stat-lock-view"> + <structname>pg_stat_lock</structname></link> for details. + </entry> + </row> + <row> <entry><structname>pg_stat_replication_slots</structname><indexterm><primary>pg_stat_replication_slots</primary></indexterm></entry> <entry>One row per replication slot, showing statistics about the @@ -3087,6 +3096,142 @@ description | Waiting for a newly initialized WAL file to reach durable storage </sect2> + + <sect2 id="monitoring-pg-stat-lock-view"> + <title><structname>pg_stat_lock</structname></title> + + <indexterm> + <primary>pg_stat_lock</primary> + </indexterm> + + <para> + The <structname>pg_stat_lock</structname> view will contain one row for each + lock type, showing cluster-wide locks statistics. + </para> + + <table id="pg-stat-lock-view" xreflabel="pg_stat_lock"> + <title><structname>pg_stat_lock</structname> View</title> + <tgroup cols="1"> + <thead> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + Column Type + </para> + <para> + Description + </para> + </entry> + </row> + </thead> + <tbody> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>locktype</structfield> <type>text</type> + </para> + <para> + Type of the lockable object: + <literal>relation</literal>, + <literal>extend</literal>, + <literal>frozenid</literal>, + <literal>page</literal>, + <literal>tuple</literal>, + <literal>transactionid</literal>, + <literal>virtualxid</literal>, + <literal>spectoken</literal>, + <literal>object</literal>, + <literal>userlock</literal>, + <literal>advisory</literal>, or + <literal>applytransaction</literal>. + (See also <xref linkend="wait-event-lock-table"/>.) + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>requests</structfield> <type>bigint</type> + </para> + <para> + Number of requests for this lock type. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>waits</structfield> <type>bigint</type> + </para> + <para> + Number of times requests for this lock type had to wait. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>timeouts</structfield> <type>bigint</type> + </para> + <para> + Number of times requests for this lock type had to wait longer + than <varname>lock_timeout</varname>. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>deadlock_timeouts</structfield> <type>bigint</type> + </para> + <para> + Number of times requests for this lock type had to wait longer + than <varname>deadlock_timeout</varname>. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>deadlocks</structfield> <type>bigint</type> + </para> + <para> + Number of times a deadlock occurred on this lock type. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>fastpath</structfield> <type>bigint</type> + </para> + <para> + Number of times this lock type was taken via fast path. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>stats_reset</structfield> <type>timestamp with time zone</type> + </para> + <para> + Time at which these statistics were last reset. + </para> + </entry> + </row> + </tbody> + </tgroup> + </table> + </sect2> + <sect2 id="monitoring-pg-stat-bgwriter-view"> <title><structname>pg_stat_bgwriter</structname></title> @@ -5158,6 +5303,12 @@ description | Waiting for a newly initialized WAL file to reach durable storage <structname>pg_stat_io</structname> view. </para> </listitem> + <listitem> + <para> + <literal>lock</literal>: Reset all the counters shown in the + <structname>pg_stat_lock</structname> view. + </para> + </listitem> <listitem> <para> <literal>recovery_prefetch</literal>: Reset all the counters shown in diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 0a0f95f6bb9..43969783069 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -964,6 +964,18 @@ CREATE VIEW pg_stat_slru AS s.stats_reset FROM pg_stat_get_slru() s; +CREATE VIEW pg_stat_lock AS + SELECT + l.locktype, + l.requests, + l.waits, + l.timeouts, + l.deadlock_timeouts, + l.deadlocks, + l.fastpath, + l.stats_reset + FROM pg_stat_get_lock() l; + CREATE VIEW pg_stat_wal_receiver AS SELECT s.pid, diff --git a/src/backend/utils/activity/pgstat_lock.c b/src/backend/utils/activity/pgstat_lock.c index fb4ed5c7362..73bdeb5ff66 100644 --- a/src/backend/utils/activity/pgstat_lock.c +++ b/src/backend/utils/activity/pgstat_lock.c @@ -21,6 +21,14 @@ static PgStat_PendingLock PendingLockStats; static bool have_lockstats = false; +PgStat_Lock * +pgstat_fetch_stat_lock(void) +{ + pgstat_snapshot_fixed(PGSTAT_KIND_LOCK); + + return &pgStatLocal.snapshot.lock; +} + /* * Simpler wrapper of pgstat_lock_flush_cb() */ diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index ef6fffe60b9..602228263a9 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1725,6 +1725,42 @@ pg_stat_get_wal(PG_FUNCTION_ARGS) wal_stats->stat_reset_timestamp)); } +Datum +pg_stat_get_lock(PG_FUNCTION_ARGS) +{ +#define PG_STAT_LOCK_COLS 8 + ReturnSetInfo *rsinfo; + PgStat_Lock *lock_stats; + + InitMaterializedSRF(fcinfo, 0); + rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; + + lock_stats = pgstat_fetch_stat_lock(); + + for (int lcktype = 0; lcktype <= LOCKTAG_LAST_TYPE; lcktype++) + { + const char *locktypename; + Datum values[PG_STAT_LOCK_COLS] = {0}; + bool nulls[PG_STAT_LOCK_COLS] = {0}; + PgStat_LockEntry *lck_stats = &lock_stats->stats[lcktype]; + + locktypename = LockTagTypeNames[lcktype]; + + values[0] = CStringGetTextDatum(locktypename); + values[1] = Int64GetDatum(lck_stats->requests); + values[2] = Int64GetDatum(lck_stats->waits); + values[3] = Int64GetDatum(lck_stats->timeouts); + values[4] = Int64GetDatum(lck_stats->deadlock_timeouts); + values[5] = Int64GetDatum(lck_stats->deadlocks); + values[6] = Int64GetDatum(lck_stats->fastpath); + values[7] = TimestampTzGetDatum(lock_stats->stat_reset_timestamp); + + tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls); + } + + return (Datum) 0; +} + /* * Returns statistics of SLRU caches. */ @@ -1909,6 +1945,7 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) pgstat_reset_of_kind(PGSTAT_KIND_BGWRITER); pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER); pgstat_reset_of_kind(PGSTAT_KIND_IO); + pgstat_reset_of_kind(PGSTAT_KIND_LOCK); XLogPrefetchResetStats(); pgstat_reset_of_kind(PGSTAT_KIND_SLRU); pgstat_reset_of_kind(PGSTAT_KIND_WAL); @@ -1926,6 +1963,8 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER); else if (strcmp(target, "io") == 0) pgstat_reset_of_kind(PGSTAT_KIND_IO); + else if (strcmp(target, "lock") == 0) + pgstat_reset_of_kind(PGSTAT_KIND_LOCK); else if (strcmp(target, "recovery_prefetch") == 0) XLogPrefetchResetStats(); else if (strcmp(target, "slru") == 0) diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index fd9448ec7b9..bb466f9e5dd 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6017,6 +6017,15 @@ proargnames => '{backend_type,object,context,reads,read_bytes,read_time,writes,write_bytes,write_time,writebacks,writeback_time,extends,extend_bytes,extend_time,hits,evictions,reuses,fsyncs,fsync_time,stats_reset}', prosrc => 'pg_stat_get_io' }, +{ oid => '9375', descr => 'statistics: per lock type statistics', + proname => 'pg_stat_get_lock', prorows => '10', proretset => 't', + provolatile => 'v', proparallel => 'r', prorettype => 'record', + proargtypes => '', + proallargtypes => '{text,int8,int8,int8,int8,int8,int8,timestamptz}', + proargmodes => '{o,o,o,o,o,o,o,o}', + proargnames => '{locktype,requests,waits,timeouts,deadlock_timeouts,deadlocks,fastpath,stats_reset}', + prosrc => 'pg_stat_get_lock' }, + { oid => '6386', descr => 'statistics: backend IO statistics', proname => 'pg_stat_get_backend_io', prorows => '5', proretset => 't', provolatile => 'v', proparallel => 'r', prorettype => 'record', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 913517ba612..38a9a7a9976 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -647,6 +647,7 @@ extern void pgstat_count_lock_timeouts(uint8 locktag_type); extern void pgstat_count_lock_deadlock_timeouts(uint8 locktag_type); extern void pgstat_count_lock_deadlocks(uint8 locktag_type); extern void pgstat_count_lock_fastpath(uint8 locktag_type); +extern PgStat_Lock *pgstat_fetch_stat_lock(void); /* * Functions in pgstat_database.c diff --git a/src/test/isolation/expected/deadlock-hard.out b/src/test/isolation/expected/deadlock-hard.out index 460653f2b86..ff448da5299 100644 --- a/src/test/isolation/expected/deadlock-hard.out +++ b/src/test/isolation/expected/deadlock-hard.out @@ -1,6 +1,12 @@ Parsed test spec with 8 sessions -starting permutation: s1a1 s2a2 s3a3 s4a4 s5a5 s6a6 s7a7 s8a8 s1a2 s2a3 s3a4 s4a5 s5a6 s6a7 s7a8 s8a1 s8c s7c s6c s5c s4c s3c s2c s1c +starting permutation: s1rl s1a1 s2a2 s3a3 s4a4 s5a5 s6a6 s7a7 s8a8 s1a2 s2a3 s3a4 s4a5 s5a6 s6a7 s7a8 s8a1 s8c s8f s7c s6c s5c s4c s3c s2c s1c s1sl +step s1rl: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + step s1a1: LOCK TABLE a1; step s2a2: LOCK TABLE a2; step s3a3: LOCK TABLE a3; @@ -21,6 +27,12 @@ step s8a1: <... completed> ERROR: deadlock detected step s7a8: <... completed> step s8c: COMMIT; +step s8f: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + step s7c: COMMIT; step s6a7: <... completed> step s6c: COMMIT; @@ -34,3 +46,9 @@ step s2a3: <... completed> step s2c: COMMIT; step s1a2: <... completed> step s1c: COMMIT; +step s1sl: SELECT deadlocks > 0, deadlock_timeouts > 0 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column? +--------+-------- +t |t +(1 row) + diff --git a/src/test/isolation/specs/deadlock-hard.spec b/src/test/isolation/specs/deadlock-hard.spec index 60bedca237a..6e8330662b4 100644 --- a/src/test/isolation/specs/deadlock-hard.spec +++ b/src/test/isolation/specs/deadlock-hard.spec @@ -25,6 +25,8 @@ setup { BEGIN; SET deadlock_timeout = '100s'; } step s1a1 { LOCK TABLE a1; } step s1a2 { LOCK TABLE a2; } step s1c { COMMIT; } +step s1sl { SELECT deadlocks > 0, deadlock_timeouts > 0 FROM pg_stat_lock WHERE locktype = 'relation'; } +step s1rl { SELECT pg_stat_reset_shared('lock'); } session s2 setup { BEGIN; SET deadlock_timeout = '100s'; } @@ -67,6 +69,7 @@ setup { BEGIN; SET deadlock_timeout = '10ms'; } step s8a8 { LOCK TABLE a8; } step s8a1 { LOCK TABLE a1; } step s8c { COMMIT; } +step s8f { SELECT pg_stat_force_next_flush(); } # Note: when s8a1 detects the deadlock and fails, s7a8 is released, making # it timing-dependent which query completion is received first by the tester. @@ -76,4 +79,4 @@ step s8c { COMMIT; } # dummy blocking mark to s8a1 to ensure it will be reported as "waiting" # regardless of that. -permutation s1a1 s2a2 s3a3 s4a4 s5a5 s6a6 s7a7 s8a8 s1a2 s2a3 s3a4 s4a5 s5a6 s6a7 s7a8(s8a1) s8a1(*) s8c s7c s6c s5c s4c s3c s2c s1c +permutation s1rl s1a1 s2a2 s3a3 s4a4 s5a5 s6a6 s7a7 s8a8 s1a2 s2a3 s3a4 s4a5 s5a6 s6a7 s7a8(s8a1) s8a1(*) s8c s8f s7c s6c s5c s4c s3c s2c s1c s1sl diff --git a/src/test/regress/expected/advisory_lock.out b/src/test/regress/expected/advisory_lock.out index 02e07765ac2..fdaa1756ba4 100644 --- a/src/test/regress/expected/advisory_lock.out +++ b/src/test/regress/expected/advisory_lock.out @@ -2,6 +2,12 @@ -- ADVISORY LOCKS -- SELECT oid AS datoid FROM pg_database WHERE datname = current_database() \gset +SELECT pg_stat_reset_shared('lock'); + pg_stat_reset_shared +---------------------- + +(1 row) + BEGIN; SELECT pg_advisory_xact_lock(1), pg_advisory_xact_lock_shared(2), @@ -48,6 +54,12 @@ WARNING: you don't own a lock of type ShareLock f | f | f | f (1 row) +SELECT pg_stat_force_next_flush(); + pg_stat_force_next_flush +-------------------------- + +(1 row) + -- automatically release xact locks at commit COMMIT; SELECT count(*) FROM pg_locks WHERE locktype = 'advisory' AND database = :datoid; @@ -56,6 +68,12 @@ SELECT count(*) FROM pg_locks WHERE locktype = 'advisory' AND database = :datoid 0 (1 row) +SELECT requests FROM pg_stat_lock WHERE locktype = 'advisory'; + requests +---------- + 4 +(1 row) + BEGIN; -- holding both session and xact locks on the same objects, xact first SELECT diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index 4286c266e17..304d9f72ed5 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1949,6 +1949,15 @@ pg_stat_io| SELECT backend_type, fsync_time, stats_reset FROM pg_stat_get_io() b(backend_type, object, context, reads, read_bytes, read_time, writes, write_bytes, write_time, writebacks, writeback_time, extends, extend_bytes, extend_time, hits, evictions, reuses, fsyncs, fsync_time, stats_reset); +pg_stat_lock| SELECT locktype, + requests, + waits, + timeouts, + deadlock_timeouts, + deadlocks, + fastpath, + stats_reset + FROM pg_stat_get_lock() l(locktype, requests, waits, timeouts, deadlock_timeouts, deadlocks, fastpath, stats_reset); pg_stat_progress_analyze| SELECT s.pid, s.datid, d.datname, diff --git a/src/test/regress/sql/advisory_lock.sql b/src/test/regress/sql/advisory_lock.sql index 8513ab8e98f..f1bff60fd37 100644 --- a/src/test/regress/sql/advisory_lock.sql +++ b/src/test/regress/sql/advisory_lock.sql @@ -4,6 +4,8 @@ SELECT oid AS datoid FROM pg_database WHERE datname = current_database() \gset +SELECT pg_stat_reset_shared('lock'); + BEGIN; SELECT @@ -26,12 +28,14 @@ SELECT pg_advisory_unlock(1), pg_advisory_unlock_shared(2), pg_advisory_unlock(1, 1), pg_advisory_unlock_shared(2, 2); +SELECT pg_stat_force_next_flush(); -- automatically release xact locks at commit COMMIT; SELECT count(*) FROM pg_locks WHERE locktype = 'advisory' AND database = :datoid; +SELECT requests FROM pg_stat_lock WHERE locktype = 'advisory'; BEGIN; -- 2.34.1 --u5iTJ95dbl+YppYN-- ^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v3 2/2] Add the pg_stat_lock view @ 2025-07-31 09:35 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 14+ messages in thread From: Bertrand Drouvot @ 2025-07-31 09:35 UTC (permalink / raw) This new view reports lock statistics. This commit also adds documentation and a few tests. XXX: Bump catversion --- doc/src/sgml/monitoring.sgml | 151 ++++++++++++++++++ src/backend/catalog/system_views.sql | 12 ++ src/backend/utils/activity/pgstat_lock.c | 8 + src/backend/utils/adt/pgstatfuncs.c | 39 +++++ src/include/catalog/pg_proc.dat | 9 ++ src/include/pgstat.h | 1 + src/test/isolation/expected/deadlock-hard.out | 20 ++- src/test/isolation/specs/deadlock-hard.spec | 5 +- src/test/regress/expected/advisory_lock.out | 18 +++ src/test/regress/expected/rules.out | 9 ++ src/test/regress/sql/advisory_lock.sql | 4 + 11 files changed, 274 insertions(+), 2 deletions(-) 51.5% doc/src/sgml/ 3.3% src/backend/catalog/ 14.3% src/backend/utils/adt/ 5.7% src/include/catalog/ 8.4% src/test/isolation/expected/ 5.9% src/test/isolation/specs/ 6.7% src/test/regress/expected/ 3.8% src/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index b77d189a500..927b62cf649 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -493,6 +493,15 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser </entry> </row> + <row> + <entry><structname>pg_stat_lock</structname><indexterm><primary>pg_stat_lock</primary></indexterm></entry> + <entry> + One row for each lock type, containing cluster-wide locks statistics. + See <link linkend="monitoring-pg-stat-lock-view"> + <structname>pg_stat_lock</structname></link> for details. + </entry> + </row> + <row> <entry><structname>pg_stat_replication_slots</structname><indexterm><primary>pg_stat_replication_slots</primary></indexterm></entry> <entry>One row per replication slot, showing statistics about the @@ -3124,6 +3133,142 @@ description | Waiting for a newly initialized WAL file to reach durable storage </sect2> + + <sect2 id="monitoring-pg-stat-lock-view"> + <title><structname>pg_stat_lock</structname></title> + + <indexterm> + <primary>pg_stat_lock</primary> + </indexterm> + + <para> + The <structname>pg_stat_lock</structname> view will contain one row for each + lock type, showing cluster-wide locks statistics. + </para> + + <table id="pg-stat-lock-view" xreflabel="pg_stat_lock"> + <title><structname>pg_stat_lock</structname> View</title> + <tgroup cols="1"> + <thead> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + Column Type + </para> + <para> + Description + </para> + </entry> + </row> + </thead> + <tbody> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>locktype</structfield> <type>text</type> + </para> + <para> + Type of the lockable object: + <literal>relation</literal>, + <literal>extend</literal>, + <literal>frozenid</literal>, + <literal>page</literal>, + <literal>tuple</literal>, + <literal>transactionid</literal>, + <literal>virtualxid</literal>, + <literal>spectoken</literal>, + <literal>object</literal>, + <literal>userlock</literal>, + <literal>advisory</literal>, or + <literal>applytransaction</literal>. + (See also <xref linkend="wait-event-lock-table"/>.) + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>requests</structfield> <type>bigint</type> + </para> + <para> + Number of requests for this lock type. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>waits</structfield> <type>bigint</type> + </para> + <para> + Number of times requests for this lock type had to wait. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>timeouts</structfield> <type>bigint</type> + </para> + <para> + Number of times requests for this lock type had to wait longer + than <varname>lock_timeout</varname>. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>deadlock_timeouts</structfield> <type>bigint</type> + </para> + <para> + Number of times requests for this lock type had to wait longer + than <varname>deadlock_timeout</varname>. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>deadlocks</structfield> <type>bigint</type> + </para> + <para> + Number of times a deadlock occurred on this lock type. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>fastpath</structfield> <type>bigint</type> + </para> + <para> + Number of times this lock type was taken via fast path. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>stats_reset</structfield> <type>timestamp with time zone</type> + </para> + <para> + Time at which these statistics were last reset. + </para> + </entry> + </row> + </tbody> + </tgroup> + </table> + </sect2> + <sect2 id="monitoring-pg-stat-bgwriter-view"> <title><structname>pg_stat_bgwriter</structname></title> @@ -5195,6 +5340,12 @@ description | Waiting for a newly initialized WAL file to reach durable storage <structname>pg_stat_io</structname> view. </para> </listitem> + <listitem> + <para> + <literal>lock</literal>: Reset all the counters shown in the + <structname>pg_stat_lock</structname> view. + </para> + </listitem> <listitem> <para> <literal>recovery_prefetch</literal>: Reset all the counters shown in diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 1ea8f1faa9e..e30c25ced31 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -985,6 +985,18 @@ CREATE VIEW pg_stat_slru AS s.stats_reset FROM pg_stat_get_slru() s; +CREATE VIEW pg_stat_lock AS + SELECT + l.locktype, + l.requests, + l.waits, + l.timeouts, + l.deadlock_timeouts, + l.deadlocks, + l.fastpath, + l.stats_reset + FROM pg_stat_get_lock() l; + CREATE VIEW pg_stat_wal_receiver AS SELECT s.pid, diff --git a/src/backend/utils/activity/pgstat_lock.c b/src/backend/utils/activity/pgstat_lock.c index fb4ed5c7362..73bdeb5ff66 100644 --- a/src/backend/utils/activity/pgstat_lock.c +++ b/src/backend/utils/activity/pgstat_lock.c @@ -21,6 +21,14 @@ static PgStat_PendingLock PendingLockStats; static bool have_lockstats = false; +PgStat_Lock * +pgstat_fetch_stat_lock(void) +{ + pgstat_snapshot_fixed(PGSTAT_KIND_LOCK); + + return &pgStatLocal.snapshot.lock; +} + /* * Simpler wrapper of pgstat_lock_flush_cb() */ diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 73ca0bb0b7f..4ca4a3f3b9b 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1725,6 +1725,42 @@ pg_stat_get_wal(PG_FUNCTION_ARGS) wal_stats->stat_reset_timestamp)); } +Datum +pg_stat_get_lock(PG_FUNCTION_ARGS) +{ +#define PG_STAT_LOCK_COLS 8 + ReturnSetInfo *rsinfo; + PgStat_Lock *lock_stats; + + InitMaterializedSRF(fcinfo, 0); + rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; + + lock_stats = pgstat_fetch_stat_lock(); + + for (int lcktype = 0; lcktype <= LOCKTAG_LAST_TYPE; lcktype++) + { + const char *locktypename; + Datum values[PG_STAT_LOCK_COLS] = {0}; + bool nulls[PG_STAT_LOCK_COLS] = {0}; + PgStat_LockEntry *lck_stats = &lock_stats->stats[lcktype]; + + locktypename = LockTagTypeNames[lcktype]; + + values[0] = CStringGetTextDatum(locktypename); + values[1] = Int64GetDatum(lck_stats->requests); + values[2] = Int64GetDatum(lck_stats->waits); + values[3] = Int64GetDatum(lck_stats->timeouts); + values[4] = Int64GetDatum(lck_stats->deadlock_timeouts); + values[5] = Int64GetDatum(lck_stats->deadlocks); + values[6] = Int64GetDatum(lck_stats->fastpath); + values[7] = TimestampTzGetDatum(lock_stats->stat_reset_timestamp); + + tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls); + } + + return (Datum) 0; +} + /* * Returns statistics of SLRU caches. */ @@ -1909,6 +1945,7 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) pgstat_reset_of_kind(PGSTAT_KIND_BGWRITER); pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER); pgstat_reset_of_kind(PGSTAT_KIND_IO); + pgstat_reset_of_kind(PGSTAT_KIND_LOCK); XLogPrefetchResetStats(); pgstat_reset_of_kind(PGSTAT_KIND_SLRU); pgstat_reset_of_kind(PGSTAT_KIND_WAL); @@ -1926,6 +1963,8 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER); else if (strcmp(target, "io") == 0) pgstat_reset_of_kind(PGSTAT_KIND_IO); + else if (strcmp(target, "lock") == 0) + pgstat_reset_of_kind(PGSTAT_KIND_LOCK); else if (strcmp(target, "recovery_prefetch") == 0) XLogPrefetchResetStats(); else if (strcmp(target, "slru") == 0) diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 83f6501df38..a0c47f6888c 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6026,6 +6026,15 @@ proargnames => '{backend_type,object,context,reads,read_bytes,read_time,writes,write_bytes,write_time,writebacks,writeback_time,extends,extend_bytes,extend_time,hits,evictions,reuses,fsyncs,fsync_time,stats_reset}', prosrc => 'pg_stat_get_io' }, +{ oid => '9375', descr => 'statistics: per lock type statistics', + proname => 'pg_stat_get_lock', prorows => '10', proretset => 't', + provolatile => 'v', proparallel => 'r', prorettype => 'record', + proargtypes => '', + proallargtypes => '{text,int8,int8,int8,int8,int8,int8,timestamptz}', + proargmodes => '{o,o,o,o,o,o,o,o}', + proargnames => '{locktype,requests,waits,timeouts,deadlock_timeouts,deadlocks,fastpath,stats_reset}', + prosrc => 'pg_stat_get_lock' }, + { oid => '6386', descr => 'statistics: backend IO statistics', proname => 'pg_stat_get_backend_io', prorows => '5', proretset => 't', provolatile => 'v', proparallel => 'r', prorettype => 'record', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index cbe226b9a07..c16c3335755 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -647,6 +647,7 @@ extern void pgstat_count_lock_timeouts(uint8 locktag_type); extern void pgstat_count_lock_deadlock_timeouts(uint8 locktag_type); extern void pgstat_count_lock_deadlocks(uint8 locktag_type); extern void pgstat_count_lock_fastpath(uint8 locktag_type); +extern PgStat_Lock *pgstat_fetch_stat_lock(void); /* * Functions in pgstat_database.c diff --git a/src/test/isolation/expected/deadlock-hard.out b/src/test/isolation/expected/deadlock-hard.out index 460653f2b86..ff448da5299 100644 --- a/src/test/isolation/expected/deadlock-hard.out +++ b/src/test/isolation/expected/deadlock-hard.out @@ -1,6 +1,12 @@ Parsed test spec with 8 sessions -starting permutation: s1a1 s2a2 s3a3 s4a4 s5a5 s6a6 s7a7 s8a8 s1a2 s2a3 s3a4 s4a5 s5a6 s6a7 s7a8 s8a1 s8c s7c s6c s5c s4c s3c s2c s1c +starting permutation: s1rl s1a1 s2a2 s3a3 s4a4 s5a5 s6a6 s7a7 s8a8 s1a2 s2a3 s3a4 s4a5 s5a6 s6a7 s7a8 s8a1 s8c s8f s7c s6c s5c s4c s3c s2c s1c s1sl +step s1rl: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + step s1a1: LOCK TABLE a1; step s2a2: LOCK TABLE a2; step s3a3: LOCK TABLE a3; @@ -21,6 +27,12 @@ step s8a1: <... completed> ERROR: deadlock detected step s7a8: <... completed> step s8c: COMMIT; +step s8f: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + step s7c: COMMIT; step s6a7: <... completed> step s6c: COMMIT; @@ -34,3 +46,9 @@ step s2a3: <... completed> step s2c: COMMIT; step s1a2: <... completed> step s1c: COMMIT; +step s1sl: SELECT deadlocks > 0, deadlock_timeouts > 0 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column? +--------+-------- +t |t +(1 row) + diff --git a/src/test/isolation/specs/deadlock-hard.spec b/src/test/isolation/specs/deadlock-hard.spec index 60bedca237a..6e8330662b4 100644 --- a/src/test/isolation/specs/deadlock-hard.spec +++ b/src/test/isolation/specs/deadlock-hard.spec @@ -25,6 +25,8 @@ setup { BEGIN; SET deadlock_timeout = '100s'; } step s1a1 { LOCK TABLE a1; } step s1a2 { LOCK TABLE a2; } step s1c { COMMIT; } +step s1sl { SELECT deadlocks > 0, deadlock_timeouts > 0 FROM pg_stat_lock WHERE locktype = 'relation'; } +step s1rl { SELECT pg_stat_reset_shared('lock'); } session s2 setup { BEGIN; SET deadlock_timeout = '100s'; } @@ -67,6 +69,7 @@ setup { BEGIN; SET deadlock_timeout = '10ms'; } step s8a8 { LOCK TABLE a8; } step s8a1 { LOCK TABLE a1; } step s8c { COMMIT; } +step s8f { SELECT pg_stat_force_next_flush(); } # Note: when s8a1 detects the deadlock and fails, s7a8 is released, making # it timing-dependent which query completion is received first by the tester. @@ -76,4 +79,4 @@ step s8c { COMMIT; } # dummy blocking mark to s8a1 to ensure it will be reported as "waiting" # regardless of that. -permutation s1a1 s2a2 s3a3 s4a4 s5a5 s6a6 s7a7 s8a8 s1a2 s2a3 s3a4 s4a5 s5a6 s6a7 s7a8(s8a1) s8a1(*) s8c s7c s6c s5c s4c s3c s2c s1c +permutation s1rl s1a1 s2a2 s3a3 s4a4 s5a5 s6a6 s7a7 s8a8 s1a2 s2a3 s3a4 s4a5 s5a6 s6a7 s7a8(s8a1) s8a1(*) s8c s8f s7c s6c s5c s4c s3c s2c s1c s1sl diff --git a/src/test/regress/expected/advisory_lock.out b/src/test/regress/expected/advisory_lock.out index 02e07765ac2..fdaa1756ba4 100644 --- a/src/test/regress/expected/advisory_lock.out +++ b/src/test/regress/expected/advisory_lock.out @@ -2,6 +2,12 @@ -- ADVISORY LOCKS -- SELECT oid AS datoid FROM pg_database WHERE datname = current_database() \gset +SELECT pg_stat_reset_shared('lock'); + pg_stat_reset_shared +---------------------- + +(1 row) + BEGIN; SELECT pg_advisory_xact_lock(1), pg_advisory_xact_lock_shared(2), @@ -48,6 +54,12 @@ WARNING: you don't own a lock of type ShareLock f | f | f | f (1 row) +SELECT pg_stat_force_next_flush(); + pg_stat_force_next_flush +-------------------------- + +(1 row) + -- automatically release xact locks at commit COMMIT; SELECT count(*) FROM pg_locks WHERE locktype = 'advisory' AND database = :datoid; @@ -56,6 +68,12 @@ SELECT count(*) FROM pg_locks WHERE locktype = 'advisory' AND database = :datoid 0 (1 row) +SELECT requests FROM pg_stat_lock WHERE locktype = 'advisory'; + requests +---------- + 4 +(1 row) + BEGIN; -- holding both session and xact locks on the same objects, xact first SELECT diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index f9bc213e5a1..8ee2e4cbea1 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1951,6 +1951,15 @@ pg_stat_io| SELECT backend_type, fsync_time, stats_reset FROM pg_stat_get_io() b(backend_type, object, context, reads, read_bytes, read_time, writes, write_bytes, write_time, writebacks, writeback_time, extends, extend_bytes, extend_time, hits, evictions, reuses, fsyncs, fsync_time, stats_reset); +pg_stat_lock| SELECT locktype, + requests, + waits, + timeouts, + deadlock_timeouts, + deadlocks, + fastpath, + stats_reset + FROM pg_stat_get_lock() l(locktype, requests, waits, timeouts, deadlock_timeouts, deadlocks, fastpath, stats_reset); pg_stat_progress_analyze| SELECT s.pid, s.datid, d.datname, diff --git a/src/test/regress/sql/advisory_lock.sql b/src/test/regress/sql/advisory_lock.sql index 8513ab8e98f..f1bff60fd37 100644 --- a/src/test/regress/sql/advisory_lock.sql +++ b/src/test/regress/sql/advisory_lock.sql @@ -4,6 +4,8 @@ SELECT oid AS datoid FROM pg_database WHERE datname = current_database() \gset +SELECT pg_stat_reset_shared('lock'); + BEGIN; SELECT @@ -26,12 +28,14 @@ SELECT pg_advisory_unlock(1), pg_advisory_unlock_shared(2), pg_advisory_unlock(1, 1), pg_advisory_unlock_shared(2, 2); +SELECT pg_stat_force_next_flush(); -- automatically release xact locks at commit COMMIT; SELECT count(*) FROM pg_locks WHERE locktype = 'advisory' AND database = :datoid; +SELECT requests FROM pg_stat_lock WHERE locktype = 'advisory'; BEGIN; -- 2.34.1 --PI+wla+m5/GDNJVD-- ^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v4 2/2] Add the pg_stat_lock view @ 2025-07-31 09:35 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 14+ messages in thread From: Bertrand Drouvot @ 2025-07-31 09:35 UTC (permalink / raw) This new view reports lock statistics. This commit also adds documentation and a few tests. XXX: Bump catversion --- doc/src/sgml/monitoring.sgml | 117 ++++++++++++++++++ src/backend/catalog/system_views.sql | 10 ++ src/backend/utils/activity/pgstat_lock.c | 8 ++ src/backend/utils/adt/pgstatfuncs.c | 40 ++++++ src/include/catalog/pg_proc.dat | 9 ++ src/include/pgstat.h | 1 + src/test/isolation/expected/deadlock-hard.out | 20 ++- src/test/isolation/specs/deadlock-hard.spec | 5 +- src/test/regress/expected/advisory_lock.out | 18 +++ src/test/regress/expected/rules.out | 7 ++ src/test/regress/sql/advisory_lock.sql | 4 + 11 files changed, 237 insertions(+), 2 deletions(-) 46.2% doc/src/sgml/ 3.2% src/backend/catalog/ 16.3% src/backend/utils/adt/ 6.2% src/include/catalog/ 9.3% src/test/isolation/expected/ 6.7% src/test/isolation/specs/ 7.3% src/test/regress/expected/ 4.4% src/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index b77d189a500..1cba84c1542 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -493,6 +493,15 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser </entry> </row> + <row> + <entry><structname>pg_stat_lock</structname><indexterm><primary>pg_stat_lock</primary></indexterm></entry> + <entry> + One row for each lock type, containing cluster-wide locks statistics. + See <link linkend="monitoring-pg-stat-lock-view"> + <structname>pg_stat_lock</structname></link> for details. + </entry> + </row> + <row> <entry><structname>pg_stat_replication_slots</structname><indexterm><primary>pg_stat_replication_slots</primary></indexterm></entry> <entry>One row per replication slot, showing statistics about the @@ -3124,6 +3133,108 @@ description | Waiting for a newly initialized WAL file to reach durable storage </sect2> + + <sect2 id="monitoring-pg-stat-lock-view"> + <title><structname>pg_stat_lock</structname></title> + + <indexterm> + <primary>pg_stat_lock</primary> + </indexterm> + + <para> + The <structname>pg_stat_lock</structname> view will contain one row for each + lock type, showing cluster-wide locks statistics. + </para> + + <table id="pg-stat-lock-view" xreflabel="pg_stat_lock"> + <title><structname>pg_stat_lock</structname> View</title> + <tgroup cols="1"> + <thead> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + Column Type + </para> + <para> + Description + </para> + </entry> + </row> + </thead> + <tbody> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>locktype</structfield> <type>text</type> + </para> + <para> + Type of the lockable object. See <link linkend="view-pg-locks"> + <structname>pg_stat_lock</structname></link> for details. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>requests</structfield> <type>bigint</type> + </para> + <para> + Number of requests for this lock type. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>timeouts</structfield> <type>bigint</type> + </para> + <para> + Number of times requests for this lock type had to wait longer + than <varname>lock_timeout</varname>. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>deadlock_timeouts</structfield> <type>bigint</type> + </para> + <para> + Number of times requests for this lock type had to wait longer + than <varname>deadlock_timeout</varname>. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>fastpath</structfield> <type>bigint</type> + </para> + <para> + Number of times this lock type was taken via fast path. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>stats_reset</structfield> <type>timestamp with time zone</type> + </para> + <para> + Time at which these statistics were last reset. + </para> + </entry> + </row> + </tbody> + </tgroup> + </table> + </sect2> + <sect2 id="monitoring-pg-stat-bgwriter-view"> <title><structname>pg_stat_bgwriter</structname></title> @@ -5195,6 +5306,12 @@ description | Waiting for a newly initialized WAL file to reach durable storage <structname>pg_stat_io</structname> view. </para> </listitem> + <listitem> + <para> + <literal>lock</literal>: Reset all the counters shown in the + <structname>pg_stat_lock</structname> view. + </para> + </listitem> <listitem> <para> <literal>recovery_prefetch</literal>: Reset all the counters shown in diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 1ea8f1faa9e..a51ae67de05 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -985,6 +985,16 @@ CREATE VIEW pg_stat_slru AS s.stats_reset FROM pg_stat_get_slru() s; +CREATE VIEW pg_stat_lock AS + SELECT + l.locktype, + l.requests, + l.timeouts, + l.deadlock_timeouts, + l.fastpath, + l.stats_reset + FROM pg_stat_get_lock() l; + CREATE VIEW pg_stat_wal_receiver AS SELECT s.pid, diff --git a/src/backend/utils/activity/pgstat_lock.c b/src/backend/utils/activity/pgstat_lock.c index 110a370b875..e26e24c260c 100644 --- a/src/backend/utils/activity/pgstat_lock.c +++ b/src/backend/utils/activity/pgstat_lock.c @@ -21,6 +21,14 @@ static PgStat_PendingLock PendingLockStats; static bool have_lockstats = false; +PgStat_Lock * +pgstat_fetch_stat_lock(void) +{ + pgstat_snapshot_fixed(PGSTAT_KIND_LOCK); + + return &pgStatLocal.snapshot.lock; +} + /* * Simpler wrapper of pgstat_lock_flush_cb() */ diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index b1df96e7b0b..6a699da8bf0 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1737,6 +1737,43 @@ pg_stat_get_wal(PG_FUNCTION_ARGS) wal_stats->stat_reset_timestamp)); } +Datum +pg_stat_get_lock(PG_FUNCTION_ARGS) +{ +#define PG_STAT_LOCK_COLS 6 + ReturnSetInfo *rsinfo; + PgStat_Lock *lock_stats; + + InitMaterializedSRF(fcinfo, 0); + rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; + + lock_stats = pgstat_fetch_stat_lock(); + + for (int lcktype = 0; lcktype <= LOCKTAG_LAST_TYPE; lcktype++) + { + const char *locktypename; + Datum values[PG_STAT_LOCK_COLS] = {0}; + bool nulls[PG_STAT_LOCK_COLS] = {0}; + PgStat_LockEntry *lck_stats = &lock_stats->stats[lcktype]; + int i = 0; + + locktypename = LockTagTypeNames[lcktype]; + + values[i++] = CStringGetTextDatum(locktypename); + values[i++] = Int64GetDatum(lck_stats->requests); + values[i++] = Int64GetDatum(lck_stats->timeouts); + values[i++] = Int64GetDatum(lck_stats->deadlock_timeouts); + values[i++] = Int64GetDatum(lck_stats->fastpath); + values[i] = TimestampTzGetDatum(lock_stats->stat_reset_timestamp); + + Assert(i + 1 == PG_STAT_LOCK_COLS); + + tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls); + } + + return (Datum) 0; +} + /* * Returns statistics of SLRU caches. */ @@ -1921,6 +1958,7 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) pgstat_reset_of_kind(PGSTAT_KIND_BGWRITER); pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER); pgstat_reset_of_kind(PGSTAT_KIND_IO); + pgstat_reset_of_kind(PGSTAT_KIND_LOCK); XLogPrefetchResetStats(); pgstat_reset_of_kind(PGSTAT_KIND_SLRU); pgstat_reset_of_kind(PGSTAT_KIND_WAL); @@ -1938,6 +1976,8 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER); else if (strcmp(target, "io") == 0) pgstat_reset_of_kind(PGSTAT_KIND_IO); + else if (strcmp(target, "lock") == 0) + pgstat_reset_of_kind(PGSTAT_KIND_LOCK); else if (strcmp(target, "recovery_prefetch") == 0) XLogPrefetchResetStats(); else if (strcmp(target, "slru") == 0) diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 83f6501df38..e208ebdbbc3 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6026,6 +6026,15 @@ proargnames => '{backend_type,object,context,reads,read_bytes,read_time,writes,write_bytes,write_time,writebacks,writeback_time,extends,extend_bytes,extend_time,hits,evictions,reuses,fsyncs,fsync_time,stats_reset}', prosrc => 'pg_stat_get_io' }, +{ oid => '9375', descr => 'statistics: per lock type statistics', + proname => 'pg_stat_get_lock', prorows => '10', proretset => 't', + provolatile => 'v', proparallel => 'r', prorettype => 'record', + proargtypes => '', + proallargtypes => '{text,int8,int8,int8,int8,timestamptz}', + proargmodes => '{o,o,o,o,o,o}', + proargnames => '{locktype,requests,timeouts,deadlock_timeouts,fastpath,stats_reset}', + prosrc => 'pg_stat_get_lock' }, + { oid => '6386', descr => 'statistics: backend IO statistics', proname => 'pg_stat_get_backend_io', prorows => '5', proretset => 't', provolatile => 'v', proparallel => 'r', prorettype => 'record', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 8460e2db159..b06a261572b 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -643,6 +643,7 @@ extern void pgstat_count_lock_requests(uint8 locktag_type); extern void pgstat_count_lock_timeouts(uint8 locktag_type); extern void pgstat_count_lock_deadlock_timeouts(uint8 locktag_type); extern void pgstat_count_lock_fastpath(uint8 locktag_type); +extern PgStat_Lock *pgstat_fetch_stat_lock(void); /* * Functions in pgstat_database.c diff --git a/src/test/isolation/expected/deadlock-hard.out b/src/test/isolation/expected/deadlock-hard.out index 460653f2b86..5ae5a8bf953 100644 --- a/src/test/isolation/expected/deadlock-hard.out +++ b/src/test/isolation/expected/deadlock-hard.out @@ -1,6 +1,12 @@ Parsed test spec with 8 sessions -starting permutation: s1a1 s2a2 s3a3 s4a4 s5a5 s6a6 s7a7 s8a8 s1a2 s2a3 s3a4 s4a5 s5a6 s6a7 s7a8 s8a1 s8c s7c s6c s5c s4c s3c s2c s1c +starting permutation: s1rl s1a1 s2a2 s3a3 s4a4 s5a5 s6a6 s7a7 s8a8 s1a2 s2a3 s3a4 s4a5 s5a6 s6a7 s7a8 s8a1 s8c s8f s7c s6c s5c s4c s3c s2c s1c s1sl +step s1rl: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + step s1a1: LOCK TABLE a1; step s2a2: LOCK TABLE a2; step s3a3: LOCK TABLE a3; @@ -21,6 +27,12 @@ step s8a1: <... completed> ERROR: deadlock detected step s7a8: <... completed> step s8c: COMMIT; +step s8f: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + step s7c: COMMIT; step s6a7: <... completed> step s6c: COMMIT; @@ -34,3 +46,9 @@ step s2a3: <... completed> step s2c: COMMIT; step s1a2: <... completed> step s1c: COMMIT; +step s1sl: SELECT deadlock_timeouts > 0 FROM pg_stat_lock WHERE locktype = 'relation'; +?column? +-------- +t +(1 row) + diff --git a/src/test/isolation/specs/deadlock-hard.spec b/src/test/isolation/specs/deadlock-hard.spec index 60bedca237a..d9ed99836e3 100644 --- a/src/test/isolation/specs/deadlock-hard.spec +++ b/src/test/isolation/specs/deadlock-hard.spec @@ -25,6 +25,8 @@ setup { BEGIN; SET deadlock_timeout = '100s'; } step s1a1 { LOCK TABLE a1; } step s1a2 { LOCK TABLE a2; } step s1c { COMMIT; } +step s1sl { SELECT deadlock_timeouts > 0 FROM pg_stat_lock WHERE locktype = 'relation'; } +step s1rl { SELECT pg_stat_reset_shared('lock'); } session s2 setup { BEGIN; SET deadlock_timeout = '100s'; } @@ -67,6 +69,7 @@ setup { BEGIN; SET deadlock_timeout = '10ms'; } step s8a8 { LOCK TABLE a8; } step s8a1 { LOCK TABLE a1; } step s8c { COMMIT; } +step s8f { SELECT pg_stat_force_next_flush(); } # Note: when s8a1 detects the deadlock and fails, s7a8 is released, making # it timing-dependent which query completion is received first by the tester. @@ -76,4 +79,4 @@ step s8c { COMMIT; } # dummy blocking mark to s8a1 to ensure it will be reported as "waiting" # regardless of that. -permutation s1a1 s2a2 s3a3 s4a4 s5a5 s6a6 s7a7 s8a8 s1a2 s2a3 s3a4 s4a5 s5a6 s6a7 s7a8(s8a1) s8a1(*) s8c s7c s6c s5c s4c s3c s2c s1c +permutation s1rl s1a1 s2a2 s3a3 s4a4 s5a5 s6a6 s7a7 s8a8 s1a2 s2a3 s3a4 s4a5 s5a6 s6a7 s7a8(s8a1) s8a1(*) s8c s8f s7c s6c s5c s4c s3c s2c s1c s1sl diff --git a/src/test/regress/expected/advisory_lock.out b/src/test/regress/expected/advisory_lock.out index 02e07765ac2..fdaa1756ba4 100644 --- a/src/test/regress/expected/advisory_lock.out +++ b/src/test/regress/expected/advisory_lock.out @@ -2,6 +2,12 @@ -- ADVISORY LOCKS -- SELECT oid AS datoid FROM pg_database WHERE datname = current_database() \gset +SELECT pg_stat_reset_shared('lock'); + pg_stat_reset_shared +---------------------- + +(1 row) + BEGIN; SELECT pg_advisory_xact_lock(1), pg_advisory_xact_lock_shared(2), @@ -48,6 +54,12 @@ WARNING: you don't own a lock of type ShareLock f | f | f | f (1 row) +SELECT pg_stat_force_next_flush(); + pg_stat_force_next_flush +-------------------------- + +(1 row) + -- automatically release xact locks at commit COMMIT; SELECT count(*) FROM pg_locks WHERE locktype = 'advisory' AND database = :datoid; @@ -56,6 +68,12 @@ SELECT count(*) FROM pg_locks WHERE locktype = 'advisory' AND database = :datoid 0 (1 row) +SELECT requests FROM pg_stat_lock WHERE locktype = 'advisory'; + requests +---------- + 4 +(1 row) + BEGIN; -- holding both session and xact locks on the same objects, xact first SELECT diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index 78a37d9fc8f..80999465baf 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1951,6 +1951,13 @@ pg_stat_io| SELECT backend_type, fsync_time, stats_reset FROM pg_stat_get_io() b(backend_type, object, context, reads, read_bytes, read_time, writes, write_bytes, write_time, writebacks, writeback_time, extends, extend_bytes, extend_time, hits, evictions, reuses, fsyncs, fsync_time, stats_reset); +pg_stat_lock| SELECT locktype, + requests, + timeouts, + deadlock_timeouts, + fastpath, + stats_reset + FROM pg_stat_get_lock() l(locktype, requests, timeouts, deadlock_timeouts, fastpath, stats_reset); pg_stat_progress_analyze| SELECT s.pid, s.datid, d.datname, diff --git a/src/test/regress/sql/advisory_lock.sql b/src/test/regress/sql/advisory_lock.sql index 8513ab8e98f..f1bff60fd37 100644 --- a/src/test/regress/sql/advisory_lock.sql +++ b/src/test/regress/sql/advisory_lock.sql @@ -4,6 +4,8 @@ SELECT oid AS datoid FROM pg_database WHERE datname = current_database() \gset +SELECT pg_stat_reset_shared('lock'); + BEGIN; SELECT @@ -26,12 +28,14 @@ SELECT pg_advisory_unlock(1), pg_advisory_unlock_shared(2), pg_advisory_unlock(1, 1), pg_advisory_unlock_shared(2, 2); +SELECT pg_stat_force_next_flush(); -- automatically release xact locks at commit COMMIT; SELECT count(*) FROM pg_locks WHERE locktype = 'advisory' AND database = :datoid; +SELECT requests FROM pg_stat_lock WHERE locktype = 'advisory'; BEGIN; -- 2.34.1 --3ZD47sYrxKE4JxfA-- ^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v6 2/2] Add the pg_stat_lock view @ 2026-02-19 10:19 Bertrand Drouvot <[email protected]> 0 siblings, 0 replies; 14+ messages in thread From: Bertrand Drouvot @ 2026-02-19 10:19 UTC (permalink / raw) This new view reports lock statistics. Note that it does not omit combinations which do not make sense (as pg_locks does). Also wait_time is reported as bigint as the deadlock_timeout default value is 1s. This commit also adds documentation and a few tests. XXX: Bump catversion --- doc/src/sgml/config.sgml | 4 +- doc/src/sgml/monitoring.sgml | 125 +++++++++++++++ src/backend/catalog/system_views.sql | 10 ++ src/backend/utils/activity/pgstat_lock.c | 8 + src/backend/utils/adt/pgstatfuncs.c | 40 +++++ src/include/catalog/pg_proc.dat | 9 ++ src/include/pgstat.h | 1 + src/test/isolation/expected/stats.out | 189 +++++++++++++++++++++++ src/test/isolation/expected/stats_1.out | 189 +++++++++++++++++++++++ src/test/isolation/specs/stats.spec | 95 ++++++++++++ src/test/regress/expected/rules.out | 7 + src/test/regress/expected/stats.out | 48 ++++++ src/test/regress/sql/stats.sql | 36 +++++ 13 files changed, 760 insertions(+), 1 deletion(-) 17.5% doc/src/sgml/ 4.9% src/backend/utils/adt/ 53.2% src/test/isolation/expected/ 11.1% src/test/isolation/specs/ 5.5% src/test/regress/expected/ 3.8% src/test/regress/sql/ 3.6% src/ diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index 0e868fb03e4..b3c353d6095 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -8854,7 +8854,9 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv; <para> Enables timing of lock waits. This parameter is on by default, as it tracks only the timings for successful acquisitions that waited longer than - <xref linkend="guc-deadlock-timeout"/>. + <xref linkend="guc-deadlock-timeout"/>. Lock timing information is + displayed in the <link linkend="monitoring-pg-stat-lock-view"> + <structname>pg_stat_lock</structname></link> view. Only superusers and users with the appropriate <literal>SET</literal> privilege can change this setting. </para> diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index b77d189a500..277275fdcb5 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -493,6 +493,15 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser </entry> </row> + <row> + <entry><structname>pg_stat_lock</structname><indexterm><primary>pg_stat_lock</primary></indexterm></entry> + <entry> + One row for each lock type, containing cluster-wide locks statistics. + See <link linkend="monitoring-pg-stat-lock-view"> + <structname>pg_stat_lock</structname></link> for details. + </entry> + </row> + <row> <entry><structname>pg_stat_replication_slots</structname><indexterm><primary>pg_stat_replication_slots</primary></indexterm></entry> <entry>One row per replication slot, showing statistics about the @@ -3124,6 +3133,116 @@ description | Waiting for a newly initialized WAL file to reach durable storage </sect2> + + <sect2 id="monitoring-pg-stat-lock-view"> + <title><structname>pg_stat_lock</structname></title> + + <indexterm> + <primary>pg_stat_lock</primary> + </indexterm> + + <para> + The <structname>pg_stat_lock</structname> view will contain one row for each + lock type, showing cluster-wide locks statistics. + </para> + + <table id="pg-stat-lock-view" xreflabel="pg_stat_lock"> + <title><structname>pg_stat_lock</structname> View</title> + <tgroup cols="1"> + <thead> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + Column Type + </para> + <para> + Description + </para> + </entry> + </row> + </thead> + <tbody> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>locktype</structfield> <type>text</type> + </para> + <para> + Type of the lockable object. See <link linkend="view-pg-locks"> + <structname>pg_locks</structname></link> for details. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>waits</structfield> <type>bigint</type> + </para> + <para> + Number of times a lock of this type had to wait because of a + conflicting lock. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>timed_waits</structfield> <type>bigint</type> + </para> + <para> + Number of times a lock of this type had to wait because of a + conflicting lock. Only incremented when <xref linkend="guc-track-lock-timing"/> + is enabled and the lock was successfully acquired after waiting longer + than <xref linkend="guc-deadlock-timeout"/>. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>wait_time</structfield> <type>bigint</type> + </para> + <para> + Total time spent waiting for locks of this type, in milliseconds. + Only incremented when <xref linkend="guc-track-lock-timing"/> is enabled and + the lock was successfully acquired after waiting longer than + <xref linkend="guc-deadlock-timeout"/>. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>fastpath_exceeded</structfield> <type>bigint</type> + </para> + <para> + Number of times a lock of this type could not be acquired via fast path + because the fast path slot limit was exceeded. You may want to increase + <xref linkend="guc-max-locks-per-transaction"/> if you feel this counter + is too high. + </para> + </entry> + </row> + + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>stats_reset</structfield> <type>timestamp with time zone</type> + </para> + <para> + Time at which these statistics were last reset. + </para> + </entry> + </row> + </tbody> + </tgroup> + </table> + </sect2> + <sect2 id="monitoring-pg-stat-bgwriter-view"> <title><structname>pg_stat_bgwriter</structname></title> @@ -5195,6 +5314,12 @@ description | Waiting for a newly initialized WAL file to reach durable storage <structname>pg_stat_io</structname> view. </para> </listitem> + <listitem> + <para> + <literal>lock</literal>: Reset all the counters shown in the + <structname>pg_stat_lock</structname> view. + </para> + </listitem> <listitem> <para> <literal>recovery_prefetch</literal>: Reset all the counters shown in diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 1ea8f1faa9e..dcf12a202e0 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -985,6 +985,16 @@ CREATE VIEW pg_stat_slru AS s.stats_reset FROM pg_stat_get_slru() s; +CREATE VIEW pg_stat_lock AS + SELECT + l.locktype, + l.waits, + l.timed_waits, + l.wait_time, + l.fastpath_exceeded, + l.stats_reset + FROM pg_stat_get_lock() l; + CREATE VIEW pg_stat_wal_receiver AS SELECT s.pid, diff --git a/src/backend/utils/activity/pgstat_lock.c b/src/backend/utils/activity/pgstat_lock.c index b410f376d49..451fba4f8d9 100644 --- a/src/backend/utils/activity/pgstat_lock.c +++ b/src/backend/utils/activity/pgstat_lock.c @@ -21,6 +21,14 @@ static PgStat_PendingLock PendingLockStats; static bool have_lockstats = false; +PgStat_Lock * +pgstat_fetch_stat_lock(void) +{ + pgstat_snapshot_fixed(PGSTAT_KIND_LOCK); + + return &pgStatLocal.snapshot.lock; +} + /* * Simpler wrapper of pgstat_lock_flush_cb() */ diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index b1df96e7b0b..3b7cbdb82be 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1737,6 +1737,43 @@ pg_stat_get_wal(PG_FUNCTION_ARGS) wal_stats->stat_reset_timestamp)); } +Datum +pg_stat_get_lock(PG_FUNCTION_ARGS) +{ +#define PG_STAT_LOCK_COLS 6 + ReturnSetInfo *rsinfo; + PgStat_Lock *lock_stats; + + InitMaterializedSRF(fcinfo, 0); + rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; + + lock_stats = pgstat_fetch_stat_lock(); + + for (int lcktype = 0; lcktype <= LOCKTAG_LAST_TYPE; lcktype++) + { + const char *locktypename; + Datum values[PG_STAT_LOCK_COLS] = {0}; + bool nulls[PG_STAT_LOCK_COLS] = {0}; + PgStat_LockEntry *lck_stats = &lock_stats->stats[lcktype]; + int i = 0; + + locktypename = LockTagTypeNames[lcktype]; + + values[i++] = CStringGetTextDatum(locktypename); + values[i++] = Int64GetDatum(lck_stats->waits); + values[i++] = Int64GetDatum(lck_stats->timed_waits); + values[i++] = Int64GetDatum(lck_stats->wait_time); + values[i++] = Int64GetDatum(lck_stats->fastpath_exceeded); + values[i] = TimestampTzGetDatum(lock_stats->stat_reset_timestamp); + + Assert(i + 1 == PG_STAT_LOCK_COLS); + + tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls); + } + + return (Datum) 0; +} + /* * Returns statistics of SLRU caches. */ @@ -1921,6 +1958,7 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) pgstat_reset_of_kind(PGSTAT_KIND_BGWRITER); pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER); pgstat_reset_of_kind(PGSTAT_KIND_IO); + pgstat_reset_of_kind(PGSTAT_KIND_LOCK); XLogPrefetchResetStats(); pgstat_reset_of_kind(PGSTAT_KIND_SLRU); pgstat_reset_of_kind(PGSTAT_KIND_WAL); @@ -1938,6 +1976,8 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) pgstat_reset_of_kind(PGSTAT_KIND_CHECKPOINTER); else if (strcmp(target, "io") == 0) pgstat_reset_of_kind(PGSTAT_KIND_IO); + else if (strcmp(target, "lock") == 0) + pgstat_reset_of_kind(PGSTAT_KIND_LOCK); else if (strcmp(target, "recovery_prefetch") == 0) XLogPrefetchResetStats(); else if (strcmp(target, "slru") == 0) diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index dac40992cbc..3cd497a8e1f 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6027,6 +6027,15 @@ proargnames => '{backend_type,object,context,reads,read_bytes,read_time,writes,write_bytes,write_time,writebacks,writeback_time,extends,extend_bytes,extend_time,hits,evictions,reuses,fsyncs,fsync_time,stats_reset}', prosrc => 'pg_stat_get_io' }, +{ oid => '9375', descr => 'statistics: per lock type statistics', + proname => 'pg_stat_get_lock', prorows => '10', proretset => 't', + provolatile => 'v', proparallel => 'r', prorettype => 'record', + proargtypes => '', + proallargtypes => '{text,int8,int8,int8,int8,timestamptz}', + proargmodes => '{o,o,o,o,o,o}', + proargnames => '{locktype,waits,timed_waits,wait_time,fastpath_exceeded,stats_reset}', + prosrc => 'pg_stat_get_lock' }, + { oid => '6386', descr => 'statistics: backend IO statistics', proname => 'pg_stat_get_backend_io', prorows => '5', proretset => 't', provolatile => 'v', proparallel => 'r', prorettype => 'record', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 615ccf540e6..39441d9698a 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -642,6 +642,7 @@ extern void pgstat_lock_flush(bool nowait); extern void pgstat_count_lock_waits(uint8 locktag_type); extern void pgstat_count_lock_fastpath_exceeded(uint8 locktag_type); extern void pgstat_count_lock_timed_wait(uint8 locktag_type, long msecs); +extern PgStat_Lock *pgstat_fetch_stat_lock(void); /* * Functions in pgstat_database.c diff --git a/src/test/isolation/expected/stats.out b/src/test/isolation/expected/stats.out index cfad309ccf3..f9cd78f03b3 100644 --- a/src/test/isolation/expected/stats.out +++ b/src/test/isolation/expected/stats.out @@ -3751,3 +3751,192 @@ test_stat_func| 1|t |t (1 row) step s1_commit: COMMIT; + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_track_lock_timing s2_set_deadlock_timeout s2_set_track_lock_timing s1_begin s1_lock_relation s2_begin s2_ff s2_lock_relation s1_sleep s1_commit s2_commit s2_report_stat_lock_relation +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_track_lock_timing: SET track_lock_timing = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_track_lock_timing: SET track_lock_timing = on; +step s1_begin: BEGIN; +step s1_lock_relation: LOCK TABLE test_stat_tab; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_relation: LOCK TABLE test_stat_tab; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_lock_relation: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_relation: SELECT waits > 0, timed_waits = waits, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column?|?column? +--------+--------+-------- +t |t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_track_lock_timing s2_set_deadlock_timeout s2_set_track_lock_timing s1_table_insert s1_begin s1_table_update_k1 s2_begin s2_ff s2_table_update_k1 s1_sleep s1_commit s2_commit s2_report_stat_lock_transactionid +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_track_lock_timing: SET track_lock_timing = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_track_lock_timing: SET track_lock_timing = on; +step s1_table_insert: INSERT INTO test_stat_tab(key, value) VALUES('k1', 1), ('k2', 1), ('k3', 1); +step s1_begin: BEGIN; +step s1_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_table_update_k1: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_transactionid: SELECT waits > 0, timed_waits = waits, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'transactionid'; +?column?|?column?|?column? +--------+--------+-------- +t |t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_track_lock_timing s2_set_deadlock_timeout s2_set_track_lock_timing s1_lock_advisory_lock s2_begin s2_ff s2_lock_advisory_lock s1_sleep s1_lock_advisory_unlock s2_lock_advisory_unlock s2_commit s2_report_stat_lock_advisory +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_track_lock_timing: SET track_lock_timing = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_track_lock_timing: SET track_lock_timing = on; +step s1_lock_advisory_lock: SELECT pg_advisory_lock(1); +pg_advisory_lock +---------------- + +(1 row) + +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_advisory_lock: SELECT pg_advisory_lock(1); <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_lock_advisory_lock: <... completed> +pg_advisory_lock +---------------- + +(1 row) + +step s2_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_commit: COMMIT; +step s2_report_stat_lock_advisory: SELECT waits > 0, timed_waits = waits, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'advisory'; +?column?|?column?|?column? +--------+--------+-------- +t |t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_track_lock_timing s2_set_deadlock_timeout s2_unset_track_lock_timing s1_begin s1_lock_relation s2_begin s2_ff s2_lock_relation s1_sleep s1_commit s2_commit s2_report_stat_lock_relation +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_track_lock_timing: SET track_lock_timing = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_unset_track_lock_timing: SET track_lock_timing = off; +step s1_begin: BEGIN; +step s1_lock_relation: LOCK TABLE test_stat_tab; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_relation: LOCK TABLE test_stat_tab; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_lock_relation: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_relation: SELECT waits > 0, timed_waits = waits, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column?|?column? +--------+--------+-------- +t |f |f +(1 row) + diff --git a/src/test/isolation/expected/stats_1.out b/src/test/isolation/expected/stats_1.out index e1d937784cb..955f6ff5ec0 100644 --- a/src/test/isolation/expected/stats_1.out +++ b/src/test/isolation/expected/stats_1.out @@ -3775,3 +3775,192 @@ test_stat_func| 1|t |t (1 row) step s1_commit: COMMIT; + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_track_lock_timing s2_set_deadlock_timeout s2_set_track_lock_timing s1_begin s1_lock_relation s2_begin s2_ff s2_lock_relation s1_sleep s1_commit s2_commit s2_report_stat_lock_relation +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_track_lock_timing: SET track_lock_timing = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_track_lock_timing: SET track_lock_timing = on; +step s1_begin: BEGIN; +step s1_lock_relation: LOCK TABLE test_stat_tab; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_relation: LOCK TABLE test_stat_tab; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_lock_relation: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_relation: SELECT waits > 0, timed_waits = waits, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column?|?column? +--------+--------+-------- +t |t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_track_lock_timing s2_set_deadlock_timeout s2_set_track_lock_timing s1_table_insert s1_begin s1_table_update_k1 s2_begin s2_ff s2_table_update_k1 s1_sleep s1_commit s2_commit s2_report_stat_lock_transactionid +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_track_lock_timing: SET track_lock_timing = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_track_lock_timing: SET track_lock_timing = on; +step s1_table_insert: INSERT INTO test_stat_tab(key, value) VALUES('k1', 1), ('k2', 1), ('k3', 1); +step s1_begin: BEGIN; +step s1_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_table_update_k1: UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1'; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_table_update_k1: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_transactionid: SELECT waits > 0, timed_waits = waits, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'transactionid'; +?column?|?column?|?column? +--------+--------+-------- +t |t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_track_lock_timing s2_set_deadlock_timeout s2_set_track_lock_timing s1_lock_advisory_lock s2_begin s2_ff s2_lock_advisory_lock s1_sleep s1_lock_advisory_unlock s2_lock_advisory_unlock s2_commit s2_report_stat_lock_advisory +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_track_lock_timing: SET track_lock_timing = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_set_track_lock_timing: SET track_lock_timing = on; +step s1_lock_advisory_lock: SELECT pg_advisory_lock(1); +pg_advisory_lock +---------------- + +(1 row) + +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_advisory_lock: SELECT pg_advisory_lock(1); <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_lock_advisory_lock: <... completed> +pg_advisory_lock +---------------- + +(1 row) + +step s2_lock_advisory_unlock: SELECT pg_advisory_unlock(1); +pg_advisory_unlock +------------------ +t +(1 row) + +step s2_commit: COMMIT; +step s2_report_stat_lock_advisory: SELECT waits > 0, timed_waits = waits, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'advisory'; +?column?|?column?|?column? +--------+--------+-------- +t |t |t +(1 row) + + +starting permutation: s1_set_deadlock_timeout s1_reset_stat_lock s1_set_track_lock_timing s2_set_deadlock_timeout s2_unset_track_lock_timing s1_begin s1_lock_relation s2_begin s2_ff s2_lock_relation s1_sleep s1_commit s2_commit s2_report_stat_lock_relation +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s1_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s1_reset_stat_lock: SELECT pg_stat_reset_shared('lock'); +pg_stat_reset_shared +-------------------- + +(1 row) + +step s1_set_track_lock_timing: SET track_lock_timing = on; +step s2_set_deadlock_timeout: SET deadlock_timeout = '10ms'; +step s2_unset_track_lock_timing: SET track_lock_timing = off; +step s1_begin: BEGIN; +step s1_lock_relation: LOCK TABLE test_stat_tab; +step s2_begin: BEGIN; +step s2_ff: SELECT pg_stat_force_next_flush(); +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_lock_relation: LOCK TABLE test_stat_tab; <waiting ...> +step s1_sleep: SELECT pg_sleep(0.5); +pg_sleep +-------- + +(1 row) + +step s1_commit: COMMIT; +step s2_lock_relation: <... completed> +step s2_commit: COMMIT; +step s2_report_stat_lock_relation: SELECT waits > 0, timed_waits = waits, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; +?column?|?column?|?column? +--------+--------+-------- +t |f |f +(1 row) + diff --git a/src/test/isolation/specs/stats.spec b/src/test/isolation/specs/stats.spec index da16710da0f..7c8e2a9214e 100644 --- a/src/test/isolation/specs/stats.spec +++ b/src/test/isolation/specs/stats.spec @@ -130,6 +130,14 @@ step s1_slru_check_stats { WHERE before.stat = 'blks_zeroed'; } +# Lock stats steps +step s1_set_deadlock_timeout { SET deadlock_timeout = '10ms'; } +step s1_set_track_lock_timing { SET track_lock_timing = on; } +step s1_reset_stat_lock { SELECT pg_stat_reset_shared('lock'); } +step s1_sleep { SELECT pg_sleep(0.5); } +step s1_lock_relation { LOCK TABLE test_stat_tab; } +step s1_lock_advisory_lock { SELECT pg_advisory_lock(1); } +step s1_lock_advisory_unlock { SELECT pg_advisory_unlock(1); } session s2 setup { SET stats_fetch_consistency = 'none'; } @@ -164,6 +172,16 @@ step s2_big_notify { SELECT pg_notify('stats_test_use', repeat(i::text, current_setting('block_size')::int / 2)) FROM generate_series(1, 3) g(i); } +# Lock stats steps +step s2_set_deadlock_timeout { SET deadlock_timeout = '10ms'; } +step s2_set_track_lock_timing { SET track_lock_timing = on; } +step s2_unset_track_lock_timing { SET track_lock_timing = off; } +step s2_report_stat_lock_relation { SELECT waits > 0, timed_waits = waits, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'relation'; } +step s2_report_stat_lock_transactionid { SELECT waits > 0, timed_waits = waits, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'transactionid'; } +step s2_report_stat_lock_advisory { SELECT waits > 0, timed_waits = waits, wait_time > 500 FROM pg_stat_lock WHERE locktype = 'advisory'; } +step s2_lock_relation { LOCK TABLE test_stat_tab; } +step s2_lock_advisory_lock { SELECT pg_advisory_lock(1); } +step s2_lock_advisory_unlock { SELECT pg_advisory_unlock(1); } ###################### # Function stats tests @@ -765,3 +783,80 @@ permutation s1_clear_snapshot s1_func_stats s1_commit + +###################### +# Lock stats tests +###################### + +# relation lock + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_track_lock_timing + s2_set_deadlock_timeout + s2_set_track_lock_timing + s1_begin + s1_lock_relation + s2_begin + s2_ff + s2_lock_relation + s1_sleep + s1_commit + s2_commit + s2_report_stat_lock_relation + +# transaction lock + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_track_lock_timing + s2_set_deadlock_timeout + s2_set_track_lock_timing + s1_table_insert + s1_begin + s1_table_update_k1 + s2_begin + s2_ff + s2_table_update_k1 + s1_sleep + s1_commit + s2_commit + s2_report_stat_lock_transactionid + +# advisory lock + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_track_lock_timing + s2_set_deadlock_timeout + s2_set_track_lock_timing + s1_lock_advisory_lock + s2_begin + s2_ff + s2_lock_advisory_lock + s1_sleep + s1_lock_advisory_unlock + s2_lock_advisory_unlock + s2_commit + s2_report_stat_lock_advisory + +# Ensure track_lock_timing behaves correctly + +permutation + s1_set_deadlock_timeout + s1_reset_stat_lock + s1_set_track_lock_timing + s2_set_deadlock_timeout + s2_unset_track_lock_timing + s1_begin + s1_lock_relation + s2_begin + s2_ff + s2_lock_relation + s1_sleep + s1_commit + s2_commit + s2_report_stat_lock_relation diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index 78a37d9fc8f..2fcb48df6c9 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1951,6 +1951,13 @@ pg_stat_io| SELECT backend_type, fsync_time, stats_reset FROM pg_stat_get_io() b(backend_type, object, context, reads, read_bytes, read_time, writes, write_bytes, write_time, writebacks, writeback_time, extends, extend_bytes, extend_time, hits, evictions, reuses, fsyncs, fsync_time, stats_reset); +pg_stat_lock| SELECT locktype, + waits, + timed_waits, + wait_time, + fastpath_exceeded, + stats_reset + FROM pg_stat_get_lock() l(locktype, waits, timed_waits, wait_time, fastpath_exceeded, stats_reset); pg_stat_progress_analyze| SELECT s.pid, s.datid, d.datname, diff --git a/src/test/regress/expected/stats.out b/src/test/regress/expected/stats.out index cd00f35bf7a..93de87747d4 100644 --- a/src/test/regress/expected/stats.out +++ b/src/test/regress/expected/stats.out @@ -1910,4 +1910,52 @@ SELECT * FROM check_estimated_rows('SELECT * FROM table_fillfactor'); (1 row) DROP TABLE table_fillfactor; +-- Test fastpath_exceeded stat +CREATE TABLE part_test (id int) PARTITION BY RANGE (id); +SELECT pg_stat_reset_shared('lock'); + pg_stat_reset_shared +---------------------- + +(1 row) + +-- Create partitions (exceeds number of slots) +DO $$ +DECLARE + max_locks int; +BEGIN + SELECT setting::int INTO max_locks + FROM pg_settings + WHERE name = 'max_locks_per_transaction'; + + FOR i IN 1..(max_locks + 10) LOOP + EXECUTE format( + 'CREATE TABLE part_test_%s PARTITION OF part_test + FOR VALUES FROM (%s) TO (%s)', + i, (i-1)*1000, i*1000 + ); + END LOOP; +END; +$$; +SELECT fastpath_exceeded AS fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation' \gset +-- Needs a lock on each partition +SELECT count(*) FROM part_test; + count +------- + 0 +(1 row) + +-- Ensure pending stats are flushed +SELECT pg_stat_force_next_flush(); + pg_stat_force_next_flush +-------------------------- + +(1 row) + +SELECT fastpath_exceeded > :fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation'; + ?column? +---------- + t +(1 row) + +DROP TABLE part_test; -- End of Stats Test diff --git a/src/test/regress/sql/stats.sql b/src/test/regress/sql/stats.sql index 8768e0f27fd..db9948eedb8 100644 --- a/src/test/regress/sql/stats.sql +++ b/src/test/regress/sql/stats.sql @@ -944,4 +944,40 @@ SELECT * FROM check_estimated_rows('SELECT * FROM table_fillfactor'); DROP TABLE table_fillfactor; +-- Test fastpath_exceeded stat +CREATE TABLE part_test (id int) PARTITION BY RANGE (id); + +SELECT pg_stat_reset_shared('lock'); + +-- Create partitions (exceeds number of slots) +DO $$ +DECLARE + max_locks int; +BEGIN + SELECT setting::int INTO max_locks + FROM pg_settings + WHERE name = 'max_locks_per_transaction'; + + FOR i IN 1..(max_locks + 10) LOOP + EXECUTE format( + 'CREATE TABLE part_test_%s PARTITION OF part_test + FOR VALUES FROM (%s) TO (%s)', + i, (i-1)*1000, i*1000 + ); + END LOOP; +END; +$$; + +SELECT fastpath_exceeded AS fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation' \gset + +-- Needs a lock on each partition +SELECT count(*) FROM part_test; + +-- Ensure pending stats are flushed +SELECT pg_stat_force_next_flush(); + +SELECT fastpath_exceeded > :fastpath_exceeded_before FROM pg_stat_lock WHERE locktype = 'relation'; + +DROP TABLE part_test; + -- End of Stats Test -- 2.34.1 --7AmA4UfQBKkkaxHa-- ^ permalink raw reply [nested|flat] 14+ messages in thread
end of thread, other threads:[~2026-02-19 10:19 UTC | newest] Thread overview: 14+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2019-06-09 20:09 [PATCH 09/10] BRIN bloom indexes Tomas Vondra <[email protected]> 2023-10-02 19:55 [PATCH v6 15/15] Trigger authors need not worry about parallelism Karl O. Pinc <[email protected]> 2023-10-02 19:55 [PATCH v7 15/16] Trigger authors need not worry about parallelism Karl O. Pinc <[email protected]> 2025-07-31 09:35 [PATCH v8 2/3] Add the pg_stat_lock view Bertrand Drouvot <[email protected]> 2025-07-31 09:35 [PATCH v5 2/2] Add the pg_stat_lock view Bertrand Drouvot <[email protected]> 2025-07-31 09:35 [PATCH v7 2/3] Add the pg_stat_lock view Bertrand Drouvot <[email protected]> 2025-07-31 09:35 [PATCH v9 2/3] Add the pg_stat_lock view Bertrand Drouvot <[email protected]> 2025-07-31 09:35 [PATCH v11 3/3] Add the pg_stat_lock view Bertrand Drouvot <[email protected]> 2025-07-31 09:35 [PATCH v10 2/2] Add the pg_stat_lock view Bertrand Drouvot <[email protected]> 2025-07-31 09:35 [PATCH v1 2/2] Add the pg_stat_lock view Bertrand Drouvot <[email protected]> 2025-07-31 09:35 [PATCH v2 2/2] Add the pg_stat_lock view Bertrand Drouvot <[email protected]> 2025-07-31 09:35 [PATCH v3 2/2] Add the pg_stat_lock view Bertrand Drouvot <[email protected]> 2025-07-31 09:35 [PATCH v4 2/2] Add the pg_stat_lock view Bertrand Drouvot <[email protected]> 2026-02-19 10:19 [PATCH v6 2/2] Add the pg_stat_lock view Bertrand Drouvot <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox